Big Data and Artificial Intelligence in Digital Finance : Increasing Personalization and Trust in Digital Finance Using Big Data and AI

This open access book presents how cutting-edge digital technologies like Big Data, Machine Learning, Artificial Intelligence (AI), and Blockchain are set to disrupt the financial sector. The book illustrates how recent advances in these technologies facilitate banks, FinTech, and financial institutions to collect, process, analyze, and fully leverage the very large amounts of data that are nowadays produced and exchanged in the sector. To this end, the book also describes some more the most popular Big Data, AI and Blockchain applications in the sector, including novel applications in the areas of Know Your Customer (KYC), Personalized Wealth Management and Asset Management, Portfolio Risk Assessment, as well as variety of novel Usage-based Insurance applications based on Internet-of-Things data. Most of the presented applications have been developed, deployed and validated in real-life digital finance settings in the context of the European Commission funded INFINITECH project, which is a flagship innovation initiative for Big Data and AI in digital finance. This book is ideal for researchers and practitioners in Big Data, AI, banking and digital finance. Preface Acknowledgments Contents Editors and Contributors About the Editors Contributors Abbreviations Part I Big Data and AI Technologies for Digital Finance 1 A Reference Architecture Model for Big Data Systems in the Finance Sector 1 Introduction 1.1 Background 1.2 Big Data Challenges in Digital Finance 1.2.1 Siloed Data and Data Fragmentation 1.2.2 Real-Time Computing 1.2.3 Mobility 1.2.4 Omni-channel Banking: Multiple Channel Management 1.2.5 Orchestration and Automation: Toward MLOps and AIOps 1.2.6 Transparency and Trustworthiness 1.3 Merits of a Reference Architecture (RA) 1.4 Chapter Structure 2 Related Work: Architectures for Systems in Banking and Digital Finance 2.1 IT Vendors' Reference Architectures 2.2 Reference Architecture for Standardization Organizations and Industrial Associations 2.3 Reference Architectures of EU Projects and Research Initiatives 2.4 Architectures for Data Pipelining 2.5 Discussion 3 The INFINITECH Reference Architecture (INFINITECH-RA) 3.1 Driving Principles: INFINITECH-RA Overview 3.2 The INFINITECH-RA 3.2.1 Logical View of the INFINITECH-RA 3.2.2 Development Considerations 3.2.3 Deployment Considerations 4 Sample Pipelines Based on the INFINITECH-RA 4.1 Simple Machine Learning Pipeline 4.2 Blockchain Data-Sharing and Analytics 4.3 Using the INFINITECH-RA for Pipeline Development and Specification 5 Conclusions References 2 Simplifying and Accelerating Data Pipelines in Digital Finance and Insurance Applications 1 Introduction 2 Challenges in Data Pipelines in Digital Finance and Insurance 2.1 IT Cost Savings 2.2 Productivity Improvements 2.3 Reduced Regulatory and Operational Risks 2.4 Delivery of New Capabilities and Services 3 Regular Data Pipeline Steps in Digital Finance and Insurance 3.1 Data Intaking 3.2 Data Transformation 3.3 Generate the Required Output 4 How LeanXcale Simplifies and Accelerates Data Pipelines 4.1 High Insertion Rates 4.2 Bidimensional Partitioning 4.3 Online Aggregates 4.4 Scalability 5 Exploring New Use Cases: The INFINITECH Approach to Data Pipelines 6 Conclusion References 3 Architectural Patterns for Data Pipelines in Digital Finance and Insurance Applications 1 Introduction 1.1 Motivation 1.2 Data Pipelining Architectural Pattern Catalogue and How LeanXcale Simplifies All of Them 2 A Taxonomy of Databases for Data Pipelining 2.1 Database Taxonomy 2.1.1 Operational Databases 2.1.2 Data Warehouses 2.1.3 Data Lakes 2.2 Operational Database Taxonomy 2.2.1 Traditional SQL Databases 2.2.2 NoSQL Databases 2.2.3 NewSQL Databases 2.3 NoSQL Database Taxonomy 2.3.1 Key-Value Data Stores 2.3.2 Document-Oriented Databases 2.3.3 Graph Databases 2.3.4 Wide-Column Data Stores 3 Architectural Patterns Dealing with Current and Historical Data 3.1 Lambda Architecture 3.2 Beyond Lambda Architecture 3.3 Current Historical Data Splitting 3.4 From Current Historical Data Splitting to Real-Time Data Warehousing 4 Architectural Patterns for Off-Loading Critical Databases 4.1 Data Warehouse Off-Loading 4.2 Simplifying Data Warehouse Off-Loading 4.3 Operational Database Off-Loading 4.4 Operational Database Off-Loading at Any Scale 4.5 Database Snapshotting 4.6 Accelerating Database Snapshotting 5 Architectural Patterns Dealing with Aggregations 5.1 In-Memory Application Aggregation 5.2 From In-Memory Application Aggregation to Online Aggregation 5.3 Detail-Aggregate View Splitting 5.4 Avoiding Detail-Aggregate View Splitting 6 Architectural Patterns Dealing with Scalability 6.1 Database Sharding 6.2 Removing Database Sharding 7 Data Pipelining in INFINITECH 8 Conclusions 4 Semantic Interoperability Framework for Digital Finance Applications 1 Introduction 2 Background: Relevant Concepts and Definitions for the INFINITECH Semantic Interoperability Framework 2.1 Interoperability 2.1.1 Semantic Interoperability 2.1.2 Semantic Models 2.1.3 Ontologies 2.1.4 Semantic Annotations 2.2 Methodologies for Ontology Engineering 2.2.1 METHONTOLOGY 2.2.2 SAMOD 2.2.3 DILIGENT 2.2.4 UPON Lite 3 INFINITECH Semantic Interoperability Framework 3.1 Methodology for Semantic Models, Ontology Engineering, and Prototyping 3.1.1 Modeling Method 3.1.2 Envisioned Roles and Functions in Semantic Models, Ontology Engineering, and Prototyping 4 Applying the Methodology: Connecting the Dots 4.1 Workflow and Technological Tools for Validation of the Methodology 4.2 Collecting 4.3 Building and Merging 4.4 Refactoring and Linking 4.4.1 Data Ingestion 4.4.2 Semantic Alignment: Building and Merging 4.4.3 Semantic Transformation: Generating a Queryable Knowledge Graphs 4.4.4 Data-Sharing/Provisioning 5 Conclusions References Part II Blockchain Technologies and Digital Currencies for Digital Finance 5 Towards Optimal Technological Solutions for Central Bank Digital Currencies 1 Understanding CBDCs 1.1 A Brief History of Definitions 1.2 How CBDCs Differ from Other Forms of Money 1.3 Wholesale and Retail CBDCs 1.4 Motivations of CBDCs 1.4.1 Financial Stability and Monetary Policy 1.4.2 Increased Competition in Payments and Threats to Financial Sovereignty 2 From Motivations to Design Options 2.1 The Design Space of CBDCs 2.2 Assessing Design Space Against Desirable Characteristics 2.2.1 Instrument Features 2.2.2 System Features References 6 Historic Overview and Future Outlook of Blockchain Interoperability 1 Multidimensional Mutually Exclusive Choices as the Source of Blockchain Limitations 2 First Attempts at Interoperability 2.1 Anchoring 2.2 Pegged Sidechains 2.3 Cross-Chain Atomic Swaps 2.4 Solution Design 3 Later Attempts at Interoperability 3.1 Polkadot 3.2 Cosmos 3.3 Interledger 3.4 Idealistic Solution Design References 7 Efficient and Accelerated KYC Using Blockchain Technologies 1 Introduction 2 Architecture 3 Use Case Scenarios 4 Sequence Diagrams 5 Implementation Solution 6 Conclusions and Future Works References 8 Leveraging Management of Customers' Consent Exploiting the Benefits of Blockchain Technology Towards SecureData Sharing 1 Introduction 2 Consent Management for Financial Services 3 Related Work 4 Methodology 4.1 User's Registration 4.2 Customer Receives a Request to Provide New Consent for Sharing His/Her Customer Data 4.3 Definition of the Consent 4.4 Signing of the Consent by the Interested Parties 4.5 Consent Form Is Stored in the Consent Management System 4.6 Consent Update or Withdrawal 4.7 Expiration of the Validity Period 4.8 Access Control Based on the Consent Forms 4.9 Retrieve Complete History of Consents 5 The INFINITECH Consent Management System 5.1 Implemented Methods 5.1.1 Definition of Consent 5.1.2 Consent Update or Withdrawal 5.1.3 Consent Expiration 5.1.4 Access Control 5.1.5 Complete History of Consents 6 Conclusions References Part III Applications of Big Data and AI in Digital Finance 9 Addressing Risk Assessments in Real-Time for Forex Trading 1 Introduction 2 Portfolio Risk 3 Risk Models 3.1 Value at Risk 3.2 Expected Shortfall 4 Real-Time Management 5 Pre-trade Analysis 6 Architecture 7 Summary References 10 Next-Generation Personalized Investment Recommendations 1 Introduction to Investment Recommendation 2 Understanding the Regulatory Environment 3 Formalizing Financial Asset Recommendation 4 Data Preparation and Curation 4.1 Why Is Data Quality Important? 4.2 Data Preparation Principles 4.3 The INFINITECH Way Towards Data Preparation 5 Approaches to Investment Recommendation 5.1 Collaborative Filtering Recommenders 5.2 User Similarity Models 5.3 Key Performance Indicator Predictors 5.4 Hybrid Recommenders 5.5 Knowledge-Based Recommenders 5.6 Association Rule Mining 6 Investment Recommendation within INFINITECH 6.1 Experimental Setup 6.2 Investment Recommendation Suitability 7 Summary and Recommendations References 11 Personalized Portfolio Optimization Using Genetic(AI) Algorithms 1 Introduction to Robo-Advisory and Algorithm-Based Asset Management for the General Public 2 Traditional Portfolio Optimization Methods 2.1 The Modern Portfolio Theory 2.2 Value at Risk (VaR) 3 Portfolio Optimization Based on Genetic Algorithms 3.1 The Concept of Evolutionary Theory 3.2 Artificial Replication Using Genetic Algorithms 3.3 Genetic Algorithms for Portfolio Optimization 3.3.1 Multiple Input Parameters 3.3.2 Data Requirements 3.3.3 A Novel and Flexible Optimization Approach Based on Genetic Algorithms 3.3.4 Fitness Factors and Fitness Score 3.3.5 Phases of the Optimization Process Utilizing Genetic Algorithms 3.3.6 Algorithm Verification 3.3.7 Sample Use Case “Sustainability” 4 Summary and Conclusions References 12 Personalized Finance Management for SMEs 1 Introduction 2 Conceptual Architecture of the Proposed Approach 3 Datasets Used and Data Enrichment 3.1 Data Utilized for the Project 3.1.1 Data Enrichment 4 Business Financial Management Tools for SMEs 4.1 Hybrid Transaction Categorization Engine 4.1.1 Rule-Based Model 4.1.2 CatBoost Classification Model 4.1.3 Explainable AI in Transaction Categorization 4.1.4 Paving the Way for Open Data Enrichment: Word Embeddings in Transaction Descriptions 4.2 Cash Flow Prediction Engine 4.3 Budget Prediction Engine 4.4 Transaction Monitoring 4.5 KPI Engine 4.6 Benchmarking Engine 4.7 Invoice Processing Invoices (Payments and Receivables) 5 Conclusion References 13 Screening Tool for Anti-money Laundering Supervision 1 Introduction 2 Related Work 2.1 Anti-money Laundering with Well-Defined Explanatory Variables 2.2 Machine Learning on Graph Structured Data 2.3 Towards Machine Learning on Graph Structured Data for AML 3 Data Ingestion and Structure 3.1 Data Enrichment and Pseudo-Anonymization 3.2 Data Storage 4 Scenario-Based Filtering 5 Automatic Detection and Warning Systems 5.1 Pattern Detection 5.2 Data Exploration Using Fourier Transforms 6 Use Cases 7 Summary References 14 Analyzing Large-Scale Blockchain Transaction Graphs for Fraudulent Activities 1 Introduction 2 Scalability Issues in Blockchain Transaction Graph Analytics 3 Distributed Data Structures of Transaction Graphs 4 BDVA Reference Model of the Blockchain Graph Analysis System 5 Parallel Fraudulent Transaction Activity Tracing 6 Tests and Results 7 Discussion and Conclusions References 15 Cybersecurity and Fraud Detection in Financial Transactions 1 Overview of Existing Financial Fraud Detection Systems and Their Limitations 2 On Artificial Intelligence-Based Fraud Detection 3 A Novel AI-Based Fraud Detection System 4 Real-Time Cybersecurity Analytics on Financial Transactions' Data Pilot in INFINITECH 5 Conclusions References Part IV Applications of Big Data and AI in Insurance 16 Risk Assessment for Personalized Health Insurance Products 1 Introduction 2 INFINITECH Healthcare Insurance System 3 Data Collection 4 INFINITECH Healthcare Insurance Pilot Test Bed 5 ML Services 5.1 Personalized Risk Assessment 5.2 Personalized Coaching 6 Conclusions References 17 Usage-Based Automotive Insurance 1 Introduction 2 Insurance Premiums for Insured Clients 3 Connected Vehicles' Infrastructures 3.1 Vehicle to Everything: V2X Communication Technologies 3.1.1 Dedicated Short-Range Communications: 802.11p 3.1.2 Long Range: Cellular Vehicle to Everything (C-V2X) 3.1.3 CTAG's Hybrid Modular Communication Unit 3.1.4 Context Information 4 Data Gathering and Homogenization Process 5 Driving Profile Model 6 Customized Car Insurance Services Powered by ML 7 Conclusions References 18 Alternative Data for Configurable and Personalized Commercial Insurance Products 1 Introduction: The State of Practice Regarding the Use of Data in the Insurance Sector 1.1 Predictive Underwriting and Automation of Underwriting Processes 1.2 Customization of Financial Product Offerings and Recommendation According to the SME's Context 1.3 Business Continuity Risk Assessment 2 Open Data in Commercial Insurance Services 2.1 Improving Current Processes 2.2 Process Automation 2.3 Ensuring Data Quality 3 Open Data for Personalization and Risk Assessment in Insurance 3.1 User Interface Application 3.2 Information Upload 3.3 Target Identification: Matching Targets with Sources 3.4 Information Gathering, Collection, and Processing: ML Algorithms 4 Alternative and Automated Insurance Risk Selection and Insurance Product Recommendation for SMEs 5 Conclusions References Part V Technologies for Regulatory Compliance in the Finance Sector 19 Large Scale Data Anonymisation for GDPR Compliance 1 Introduction 2 Anonymisation as Regulatory Compliance Tool 3 Anonymisation at Large Scale: Challenges and Solutions 4 Conclusions References 20 Overview of Applicable Regulations in Digital Finance and Supporting Technologies 1 Applicable Regulations in Digital Finance 2 Main Digital Finance Regulations 2.1 The General Data Protection Regulation (GDPR) 3 The Market in Financial Instruments Directive II (MIFID II) 3.1 Payment Services Directive 2 (PSD2) 3.2 PSD II Security Requirements/Guidelines 3.3 4th Anti-money Laundering (AMLD4) 3.4 Basel IV 3.5 EU Legislative Proposal on Markets in Crypto-Assets (MiCA) 3.6 EU Draft Digital Operational Resilience Act (DORA) 3.7 EU Draft AI Act 4 Supporting Technologies for Regulations in Digital Finance 5 Regulatory Compliance Tool and Data Protection Orchestrator (DPO) 6 Conclusions References Index "This open access book presents how cutting-edge digital technologies like Machine Learning, Artificial Intelligence (AI), and Blockchain are set to disrupt the financial sector. The book illustrates how recent advances in these technologies facilitate banks, FinTechs, and financial institutions to collect, process, analyze, and fully leverage the very large amounts of data that are nowadays produced and exchanged in the sector. To this end, the book also introduces some of the most popular Big Data, AI and Blockchain applications in the sector, including novel applications in the areas of Know Your Customer (KYC), Personalized Wealth Management and Asset Management, Portfolio Risk Assessment, as well as variety of novel Usage-based Insurance applications based on Internet-of-Things data. Most of the presented applications have been developed, deployed and validated in real-life digital finance settings in the context of the European Commission funded INFINITECH project, which is a flagship innovation initiative for Big Data and AI in digital finance. This book is ideal for researchers and practitioners in Big Data, AI, banking and digital finance. Introduces the latest advances in Big Data and AI in Digital Finance that enable scalable, effective, and real-time analytics; explains the merits of Blockchain technology in digital finance, including applications beyond the blockbuster cryptocurrencies; presents practical applications of cutting edge digital technologies in the digital finance sector; illustrates the regulatory environment of the financial sector and presents technical solutions that boost compliance to applicable regulations; this book is open access, which means that you have free and unlimited access."--Cover page 4