Machine Learning and Knowledge Discovery in Databases: Research Track: European Conference, ECML PKDD 2023, Turin, Italy, September 18–22, 2023, ... IV (Lecture Notes in Artificial Intelligence)

The multi-volume set LNAI 14169 until 14175 constitutes the refereed proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2023, which took place in Turin, Italy, in September 2023.The 196 papers were selected from the 829 submissions for the Research Track, and 58 papers were selected from the 239 submissions for the Applied Data Science Track. The volumes are organized in topical sections as follows:Part I: Active Learning; Adversarial Machine Learning; Anomaly Detection; Applications; Bayesian Methods; Causality; Clustering.Part II: ​Computer Vision; Deep Learning; Fairness; Federated Learning; Few-shot learning; Generative Models; Graph Contrastive Learning.Part III: ​Graph Neural Networks; Graphs; Interpretability; Knowledge Graphs; Large-scale Learning.Part IV: ​Natural Language Processing; Neuro/Symbolic Learning; Optimization; Recommender Systems; Reinforcement Learning; Representation Learning.Part V: ​Robustness; Time Series; Transfer and Multitask Learning.Part VI: ​Applied Machine Learning; Computational Social Sciences; Finance; Hardware and Systems; Healthcare & Bioinformatics; Human-Computer Interaction; Recommendation and Information Retrieval.​Part VII: Sustainability, Climate, and Environment.- Transportation & Urban Planning.- Demo. Preface Organization Invited Talks Abstracts Neural Wave Representations Physics-Inspired Graph Neural Networks Mapping Generative AI Contents – Part IV Natural Language Processing Unsupervised Deep Cross-Language Entity Alignment 1 Introduction 2 Related Work 2.1 Supervised Entity Alignment 2.2 Semi-supervised Entity Alignment 2.3 Unsupervised Entity Alignment 3 Proposed Method 3.1 Base Symbol Definition 3.2 Feature Embedding Module 3.3 Alignment Module 4 Experiments 4.1 Cross-Lingual Dataset 4.2 Comparative 4.3 Evaluation Indicate and Experiment Setting 4.4 Experimental Results 5 Ablation Study 5.1 Translator and Encoder Analysis 5.2 Alignment Module Analysis 5.3 Additional Analysis 6 Error Mode Analysis 7 Conclusion and Future Research References Corpus-Based Relation Extraction by Identifying and Refining Relation Patterns 1 Introduction 2 Problem Formulation 3 Methodology 3.1 Representation of Relation Triple 3.2 Initial Weak Supervision Extraction 3.3 Weak Supervision Noise Reduction via Clustering 3.4 Generalization via Prompt-Tuning 4 Experiments 4.1 Relation Extraction 4.2 Positive and Negative Samples 4.3 Cluster Visualization 4.4 Ablation Study 5 Related Work 6 Conclusions References Learning to Play Text-Based Adventure Games with Maximum Entropy Reinforcement Learning 1 Introduction 2 Related Work 3 Problem Setting and Background 3.1 SAC for Discrete Action Spaces 4 Reward Shaping Method 5 Experimental Results 5.1 Datasets 5.2 Experimental Settings 5.3 Results 6 Limitations and Future Work 7 Conclusion References SALAS: Supervised Aspect Learning Improves Abstractive Multi-document Summarization Through Aspect Information Loss 1 Introduction 2 Related Work 2.1 Multi-document Summarization 2.2 Aspect-Related Text Generation 3 Methodology 3.1 Task and Framework Formulation 3.2 Encoder Probe 3.3 Decoder Probe 3.4 Aspect-Guided Generator 3.5 Training Objective 4 Experiments 4.1 Datasets 4.2 Baselines 4.3 Implementation Details 4.4 Main Results 4.5 Results of Ablation Study 4.6 Analysis and Discussion 5 Conclusion References KL Regularized Normalization Framework for Low Resource Tasks 1 Introduction 2 Related Work 3 Theoretical Foundation of KL Regularized Normalization 3.1 Preliminaries: Batch Normalization 3.2 KL Regularized Batch Normalization 4 Experiments 4.1 Model Architecture 4.2 Comparison Methods 4.3 Training Details 4.4 Analysis of Increased Expressive Power 4.5 Analysis of out of Domain Generalization 4.6 Impact of KL-Norm on Overfitting 4.7 Analysis of Removal of Irrelevant Features 4.8 Analysis of Model Parameters 4.9 Ablation Study 5 Conclusion References Improving Autoregressive NLP Tasks via Modular Linearized Attention 1 Introduction 2 Background and Motivation 2.1 Fundamentals of Transformers and Attention 2.2 Linear Transformers 2.3 cosFormer and Re-weighting Mechanisms 2.4 Motivation for Investigation 3 Proposed Approach 3.1 Modular Linearized Attention 3.2 Augmenting CosFormer for Decoder Cross-Attention 3.3 Closing the Gap: Target Sequence Length Prediction 4 Training and Evaluation Details 4.1 Model Configurations and Training Hyperparameters 4.2 Evaluation Setup and Metrics 5 Results 5.1 TTS Training Results for Targeted Ablations 5.2 Training and Evaluation Results for Finalized TTS Configurations 5.3 en-de NMT and SimulST Training and Evaluation Results 6 Conclusion References Enhancing Table Retrieval with Dual Graph Representations 1 Introduction 2 Related Work 3 Methodology 3.1 Task Definition 3.2 Graph Construction 3.3 Dual-Graph Representation Learning 3.4 Prediction 4 Experiments 4.1 Datasets 4.2 Baselines 4.3 Implementation Details 4.4 Results 4.5 Analyses 5 Conclusion References A Few Good Sentences: Content Selection for Abstractive Text Summarization 1 Introduction 2 SWORTS Selection 2.1 Content Selection Metrics 2.2 SWORTS Selection Pipeline 2.3 COMET Variants 3 Experimental Setup 3.1 Self-training 3.2 Cross-Training 3.3 Zero-Shot Adaptation 3.4 Experimental Settings 4 Results and Analysis 4.1 Self-training 4.2 Cross-training 4.3 Zero-Shot Adaptation 4.4 Limitations and Opportunities with SWORTS Selection 5 Conclusion References Encouraging Sparsity in Neural Topic Modeling with Non-Mean-Field Inference 1 Introduction 2 Related Work 3 Approach 3.1 VAE-LDA 3.2 SpareNTM 3.3 Objective Function 3.4 Neural Network Architecture 4 Experiment 4.1 Datasets 4.2 Baseline Methods and Parameter Settings 4.3 Evaluation Metric 4.4 Experimental Results 5 Conclusion References Neuro/Symbolic Learning The Metric is the Message: Benchmarking Challenges for Neural Symbolic Regression 1 Introduction 2 Methods 2.1 Post Equation Generation Coefficient Fitting 2.2 Benchmarks 2.3 Metrics 2.4 NSR Methods 2.5 Control Equations 3 Results 3.1 Numeric Metrics 3.2 Symbolic Metrics 4 Discussion 5 Conclusion A Appendix B Ethics References Symbolic Regression via Control Variable Genetic Programming 1 Introduction 2 Preliminaries 3 Control Variable Genetic Programming 3.1 Control Variable Experiment 3.2 Control Variable Genetic Programming 3.3 Theoretical Analysis 4 Related Work 5 Experiments 5.1 Experimental Settings 5.2 Experimental Analysis 6 Conclusion References Neural Class Expression Synthesis in ALCHIQ(D) 1 Introduction 2 Related Work 3 Background 3.1 Description Logics 3.2 Refinement Operators 3.3 Class Expression Learning 3.4 Knowledge Graph Embedding 3.5 The Set Transformer Architecture 4 Proposed Approach (NCES2) 4.1 Preliminaries 4.2 Learning Problem 4.3 Encoding Positive and Negative Examples 4.4 Loss Function 4.5 Measuring Performance During Training 4.6 Class Expression Synthesis 4.7 Model Ensembling 5 Evaluation 5.1 Experimental Setup 5.2 Results and Discussion 6 Conclusion and Future Work References Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approach 1 Introduction 2 Related Works 3 Preliminaries 3.1 Logic Programming 3.2 Relational Markov Decision Process 3.3 Problem Statement 4 Proposed Approach 4.1 Rule Generation 4.2 Inference 4.3 Semantic Constraints 5 Experiments 5.1 Experimental Setup 5.2 Results 6 Conclusion References ReOnto: A Neuro-Symbolic Approach for Biomedical Relation Extraction 1 Introduction 2 Related Work 3 Problem Formulation and Approach 3.1 Symbolic Module 3.2 Encoding Module 3.3 Graph Neural Network 4 Experimental Setup 5 Results 6 Ablation Study 6.1 Effectiveness of Number of Ontologies 6.2 Effectiveness of Number of Hops 7 Case Study 8 Conclusion and Future Work References Optimization NKFAC: A Fast and Stable KFAC Optimizer for Deep Neural Networks 1 Introduction 2 Backgrounds and Preliminaries 2.1 Matrix Inverse 2.2 KFAC Algorithm 2.3 About Other Second-Order Optimizers 3 Methodology 3.1 Motivation 3.2 Newton's Iteration of Matrix Inverse 3.3 NKFAC 3.4 Implementations and AdaNKFAC 4 Experiments 4.1 Experimental Setup 4.2 Results on CIFAR100/10 4.3 Results on ImageNet 4.4 Results on COCO 5 Conclusion References Exact Combinatorial Optimization with Temporo-Attentional Graph Neural Networks 1 Introduction 2 Related Work 3 Background 4 Methodology 5 Experiments 6 Conclusion 7 Statement of Ethics References Improved Multi-label Propagation for Small Data with Multi-objective Optimization 1 Introduction 2 Theoretical Background 3 Label Propagation on Absorbing Markov Chain 3.1 Computing Transition Probabilities 4 Leveraging Label Dependence: Interpolated Label Propagation as a Stacking Method 5 Evaluation Measures and Thresholding Strategies 6 Finding Compromise Solution for Multiple Evaluation Measures 7 Experiments 7.1 Experimental Setup 7.2 Results 8 Conclusions References Fast Convergence of Random Reshuffling Under Over-Parameterization and the Polyak-Łojasiewicz Condition 1 Introduction 2 Related Work 3 Assumptions 4 Contributions 5 Convergence Results 5.1 Proof Sketch 6 Experimental Results 6.1 Synthetic Experiments 6.2 Binary Classification Using RBF Kernels 6.3 Multi-class Classification Using Deep Networks 7 Conclusion References A Scalable Solution for the Extended Multi-channel Facility Location Problem 1 Introduction 1.1 Contributions 1.2 Problem Formulation 2 Background 2.1 Submodularity 2.2 Optimal Transport (OT) 3 A Scalable Approximate Solution 3.1 Fast Value Oracle 4 Related Work 5 Experiments 6 Conclusion References Online State Exploration: Competitive Worst Case and Learning-Augmented Algorithms 1 Introduction 1.1 The Online State Exploration Problem 1.2 Applications of Online State Exploration 1.3 Our Contributions 2 Algorithms and Analysis 2.1 Algorithmic Intuition and Framework 2.2 Warm-Up: Worst Case Algorithms 2.3 Algorithms Leveraging Predictions 3 Applications of OSEP: Problem-Specific Analyses and Better Results 3.1 Online Bidding 3.2 Multi-dimensional Knapsack Cover 3.3 Multi-directional Cow Path 4 Experiments 5 Conclusion References Learning Graphical Factor Models with Riemannian Optimization 1 Introduction 2 Background and Proposed Framework 2.1 Gaussian Graphical and Related Models 2.2 Elliptical Distributions 2.3 Low-Rank Factor Models 2.4 Learning Elliptical Graphical Factor Models 3 Learning Graphs with Riemmanian Optimization 3.1 Learning GGM/EGM: Optimization on Sp++ 3.2 Learning GGFM/EGFM: Optimization on Mp,k 3.3 Algorithms Properties 4 Experiments 4.1 Validations on Synthetic Data 4.2 Real-World Data Sets 5 Conclusion References Recommender Systems ConGCN: Factorized Graph Convolutional Networks for Consensus Recommendation 1 Introduction 2 Consensus Recommendation 2.1 Problem Statement 2.2 Consensus Recommendation 3 Factorized Graph Convolutional Network 3.1 Homogeneous Effect Embedding 3.2 Heterogeneous Interaction Reconstruction 3.3 Dynamic Loss Fusion 4 Experiments 4.1 Benchmark Datasets 4.2 Homogeneous Graph Generation 4.3 Experimental Settings 4.4 Experiment Results 5 Conclusion References Long-Tail Augmented Graph Contrastive Learning for Recommendation 1 Introduction 2 Preliminaries 2.1 GCN-Based Recommendation 2.2 Long-Tail Distribution in the Graph 3 Methodology 3.1 Overview of LAGCL 3.2 Long-Tail Augmentation 3.3 Contrastive Learning 3.4 Multi-task Training 4 Experiments 4.1 Experimental Setups 4.2 Performance Comparison 4.3 Ablation Study 4.4 Further Analysis of LAGCL 5 Related Work 5.1 GCN-Based Recommendation 5.2 Self-supervised Learning in Recommender Systems 6 Conclusion References News Recommendation via Jointly Modeling Event Matching and Style Matching*-12pt 1 Introduction 2 Related Work 2.1 News Recommendation 2.2 News Event and Style Modeling 3 News Event and Style Matching Model 3.1 News Encoder 3.2 Event-Style Disentangler 3.3 Event Matching Module 3.4 Style Matching Module 3.5 News Recommendation 4 Experiment 4.1 Dataset and Experimental Settings 4.2 Comparison with Competing Methods 4.3 Ablation Study 4.4 Comparison with Different News Encoders 4.5 Hyperparameter Analysis 5 Conclusion 6 Ethics Consideration References BalancedQR: A Framework for Balanced Query Recommendation 1 Introduction 2 Related Work 3 Problem and Framework 3.1 Problem Setup 4 Proposed Method 4.1 Recommendation Framework 4.2 Our Implementation 4.3 Limitations 5 Experimental Setup 5.1 Data 5.2 Search Engines 5.3 Word Embedding 6 Analysis and Discussion 6.1 Reddit 6.2 Twitter 7 Conclusion and Future Work References Reinforcement Learning On the Distributional Convergence of Temporal Difference Learning 1 Introduction 1.1 Related Works 1.2 Contributions 2 Preliminaries 3 Distributional Convergence of TD 3.1 The I.I.D. Observation Model 3.2 The Markov Chain Observation Model 3.3 Jacobi Preconditioning Explanation 3.4 Discussions 4 Proofs 4.1 Proof of Lemma 1 4.2 Proof of Theorem 1 4.3 Proof of Proposition 1 4.4 Proof of Theorem 2 4.5 Proof of Proposition 2 5 Conclusion References Offline Reinforcement Learning with On-Policy Q-Function Regularization 1 Introduction 2 Related Works 3 Problem Formulation and Notations 4 Introduction and Evaluation of Qsarsa 4.1 Evaluation of Qsarsa 5 Offline RL with Q-sarsa (Qsarsa-AC) 5.1 Critic with Qsarsa 5.2 Actor with Qsarsa 5.3 A Variant Qsarsa-AC2 6 Experimental Evaluation 6.1 Settings and Baselines 6.2 Main Results 6.3 Additional Results 7 Conclusion References Alpha Elimination: Using Deep Reinforcement Learning to Reduce Fill-In During Sparse Matrix Decomposition 1 Introduction 2 Related Work 3 Alpha Elimination 3.1 Section1: Formulating the Game 3.2 Section 2: Applying Deep MCTS 3.3 Section3: Neural Network Architecture 4 Alpha Elimination: Details 4.1 Training 4.2 Prediction 4.3 Remark on the Role of Masking 4.4 Scaling to Larger Matrices 5 Experiments and Discussion 5.1 Experimental Setup 5.2 Experimental Results 6 Conclusion and Future Work References Learning Hierarchical Planning-Based Policies from Offline Data 1 Introduction 2 Related Work 3 Technical Background and Problem Statement 3.1 Offline Learning Setting 3.2 Hierarchical Policy Architecture 3.3 Value Refinement Network 3.4 Problem Statement 4 HORIBLe-VRN Offline Learning Algorithm 4.1 Graphical Model 4.2 Dataset Pre-Processing 4.3 Stage 1: Hierarchical IL via EM Procedure 4.4 Stage 2: Offline RL HL Policy Refinement 5 Empirical Evaluation 5.1 Environment 5.2 Data Collection 5.3 Baselines 5.4 Results and Discussion 6 Conclusion References Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes 1 Introduction 2 Related Work 3 Preliminaries 4 Meta-MDP 5 Context Lipschitz Continuity 6 Fitted Q-Iteration on Meta-MDP 7 Experimental Evaluation 8 Discussion, Limitations and Future Work References Invariant Lipschitz Bandits: A Side Observation Approach 1 Introduction 1.1 Our Contributions 1.2 Related Works 1.3 Outline 2 Preliminary and Problem Setup 2.1 Preliminary 2.2 Problem Formulation 2.3 A Warm-Up: Invariant Finite Unstructured MAB 3 UniformMesh-N Algorithm and Regret Analysis 3.1 Graph Structure Induced by the Group Action 3.2 UniformMesh-N: Uniform Discretization with the Transformation Group 4 Regret Lower Bound 5 Conclusion References Filtered Observations for Model-Based Multi-agent Reinforcement Learning 1 Introduction 2 Related Work 3 Background and Notations 3.1 Cooperative Multi-Agent Problem 3.2 Latent Imagination for Learning Policy 3.3 Guided Diffusion Models 4 Methodology 4.1 Motivation 4.2 Control with Denoised World Models 4.3 Learning Objective 4.4 Training and Sampling 5 Experiments 5.1 Toy Example 5.2 Cooperative Multi-agent Tasks on SMAC 5.3 Ablation Studies 6 Conclusion References Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning 1 Introduction 2 Related Work 3 Background 3.1 Rainbow and Its Data-Efficient Version 3.2 Transformer Layers and Self-attention 3.3 Vision Transformer and Masked Autoencoder 4 Architecture 4.1 MAE Adaptation 4.2 Salient Patch Selection 4.3 Transformed-Based RL 5 Experimental Results 6 Conclusion References Eigensubspace of Temporal-Difference Dynamics and How It Improves Value Approximation in Reinforcement Learning 1 Introduction 2 Preliminaries 3 Method 3.1 An Inherent Path of Value Approximation 3.2 Using Eigensubspace Regularization to Improve Value Approximation 3.3 Theoretical Analysis 3.4 Practical Algorithm 4 Experiments 4.1 Evaluation Setting 4.2 Performance Evaluation 4.3 Variance and Approximation Error 4.4 Ablation 5 Related Work 5.1 Value Function Approximation 5.2 Using Structural Information of MDPs 6 Conclusion References Representation Learning Learning Disentangled Discrete Representations 1 Introduction 2 Disentangled Representations 3 Learning Disentangled Discrete Representations 3.1 Neighborhoods in the Latent Space 3.2 Disentangling Properties of the Discrete VAE 3.3 Structural Advantages of the Discrete VAE 3.4 The Straight-Through Gap 4 Related Work 5 Experimental Setup 6 Experimental Results 6.1 Improvement in Unsupervised Disentanglement Properties 6.2 Match State-of-the-Art Unsupervised Disentanglement Methods 6.3 Unsupervised Selection of Models with Strong Disentanglement 6.4 Utilize Label Information to Improve Discrete Representations 6.5 Visualization of the Latent Categories 7 Conclusion References Boosting Object Representation Learning via Motion and Object Continuity 1 Introduction 2 Motion and Object Continuity 2.1 Motion Supervision 2.2 Object Continuity 2.3 General Training Scheme 3 Experimental Evaluations 4 Related Work 5 Limitations and Future Work 6 Conclusions References Learning Geometric Representations of Objects via Interaction 1 Introduction 2 Related Work 3 Formalism and Assumptions 4 Method 4.1 Representations and Equivariance 4.2 Learning the Representation 4.3 Incorporating Volumes of Objects 5 Experiments 5.1 Sprites 5.2 Soccer 5.3 Control Task 6 Conclusions and Future Work References On the Good Behaviour of Extremely Randomized Trees in Random Forest-Distance Computation 1 Introduction 2 Background 2.1 RF-Distances 3 Main Result 3.1 Step 1: The Representation 3.2 Step 2: The Bounds 3.3 The Proofs 4 Understanding the Bounds 4.1 The Number of Trees M 4.2 The Approximation 4.3 Using the Bounds for Estimating the Size of the Forest 5 Discussion and Final Remarks References Hypernetworks Build Implicit Neural Representations of Sounds 1 Introduction 2 Related Works 2.1 Implicit Neural Representations (INRs) 2.2 Generalizable INRs 2.3 Deep Neural Networks for Audio Processing 3 Method 3.1 Implicit Neural Representations 3.2 Hypernetwork 3.3 Hypernetwork Adaptation for Audio Data 3.4 Optimization for Audio Reconstruction 3.5 Alternative Ways to Leverage Hypernetworks for INR Generation 3.6 Hypernetwork Weight Initialization 4 Experiments 4.1 Setting 4.2 Reconstruction Quality with FMLP Target Network 4.3 Training SIREN Target Networks 4.4 Upper Bound of the INR Reconstruction Quality 4.5 Exploring Different Ways to Use Weights Generated by the Hypernetwork 4.6 Higher-Frequency Weighting in the Loss Function 4.7 Results on Other Datasets 4.8 Representational Capability of INRs 5 Conclusions References Contrastive Representation Through Angle and Distance Based Loss for Partial Label Learning 1 Introduction 2 Background and Problem Formulation 2.1 Partial Label Learning (PLL): Notation and Preliminaries 2.2 Literature Review 3 Motivation for Proposing CRADL Algorithm 3.1 Theoretical Analysis 3.2 Angle Based Contrastive Loss 3.3 Distance Based Contrastive Loss 4 Proposed Algorithm: CRADL 4.1 Initialization 4.2 Algorithm 4.3 Class-Prototype and Pseudo-target Update 5 Experimental Details 5.1 Model Architecture and Datasets 5.2 Implementation Details 6 Results and Analysis 6.1 Comparison with Recent PLL Methods 6.2 Ablation Study 7 Conclusion References Equivariant Representation Learning in the Presence of Stabilizers 1 Introduction 2 Related Work 3 Group Theory Background 4 Equivariant Isomorphic Networks (EquIN) 4.1 Parametrizing G via the Exponential Map 4.2 Training Objective 5 Experiments 5.1 Datasets 5.2 Comparisons 5.3 Quantitative Results 5.4 Qualitative Results 5.5 Hyperparameter Analysis 6 Conclusions and Future Work References Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis 1 Introduction 1.1 Contributions 1.2 Related Work 2 Preliminaries 2.1 Problem Setup 2.2 Reproducing Kernels 3 Kernel Contrastive Learning 4 A Formulation Based on Statistical Similarity 4.1 Key Ingredient: Similarity Function 4.2 Formulation and Example 5 Theoretical Results 5.1 KCL as Representation Learning with Statistical Similarity 5.2 A New Upper Bound of the Classification Error 5.3 A Generalization Error Bound for KCL 5.4 Application of the Theoretical Results: A New Surrogate Bound 6 Conclusion and Discussion References BipNRL: Mutual Information Maximization on Bipartite Graphs for Node Representation Learning 1 Introduction 2 Related Works 3 Methodology 3.1 Background 3.2 Mutual Information Maximization Without Readout Functions 3.3 Decoupled Graph Context Loss 3.4 Attention-Based Neighbourhood Aggregation 3.5 Biased Neighbourhood Sampling 4 Experiments 4.1 Datasets and Tasks 4.2 Baselines 4.3 Experiment Setting 4.4 Performance Evaluation 4.5 Analysis and Discussion 5 Conclusion References Author Index