Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2022, Grenoble, France, September 19–23, 2022, Proceedings, Part III (Lecture Notes in Artificial Intelligence)

The multi-volume set LNAI 13713 until 13718 constitutes the refereed proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2022, which took place in Grenoble, France, in September 2022.The 236 full papers presented in these proceedings were carefully reviewed and selected from a total of 1060 submissions. In addition, the proceedings include 17 Demo Track contributions. The volumes are organized in topical sections as follows: Part I: Clustering and dimensionality reduction; anomaly detection; interpretability and explainability; ranking and recommender systems; transfer and multitask learning; Part II: Networks and graphs; knowledge graphs; social network analysis; graph neural networks; natural language processing and text mining; conversational systems; Part III: Deep learning; robust and adversarial machine learning; generative models; computer vision; meta-learning, neural architecture search; Part IV: Reinforcement learning; multi-agent reinforcement learning; bandits and online learning; active and semi-supervised learning; private and federated learning;. Part V: Supervised learning; probabilistic inference; optimal transport; optimization; quantum, hardware; sustainability; Part VI: Time series; financial machine learning; applications; applications: transportation; demo track. Preface Organization Contents – Part III Deep Learning DialCSP: A Two-Stage Attention-Based Model for Customer Satisfaction Prediction in E-commerce Customer Service 1 Introduction 2 Related Work 3 Methodology 3.1 Problem Definition 3.2 Proposed Model 3.3 Session-Encoder 3.4 Intra-stage 3.5 Inter-stage 3.6 Session-Decoder 4 Experimental Settings 4.1 Datasets 4.2 Baselines 4.3 Hyper Parameters 5 Results and Analysis 5.1 Overall Results 5.2 Ablation Study 5.3 Case Analysis 6 Conclusion References Foveated Neural Computation 1 Introduction 2 Foveated Convolutional Layers 2.1 Learning with Attention in Foveated Neural Networks 3 Experiments 4 Conclusions and Future Work References Class-Incremental Learning via Knowledge Amalgamation 1 Introduction 2 Related Work 2.1 Domain Adaptation 2.2 Knowledge Distillation 2.3 Knowledge Amalgamation 3 Problem Formulation 4 Proposed Method 4.1 Joint Representation Learning 4.2 Soft Domain Adaptation 4.3 Final Loss 5 Experiments 5.1 Replay Memory 5.2 Baselines Setup 5.3 Metrics 5.4 Benchmarks 5.5 Numerical Results 5.6 Ablation Study 6 Conclusion References Trigger Detection for the sPHENIX Experiment via Bipartite Graph Networks with Set Transformer 1 Introduction 2 Related Work 3 Problem Definition 4 Transverse Momentum Estimation 4.1 Physics Relation Between Transverse Momentum and Track Curvature 4.2 Track Curvature Fitting 4.3 Momentum Estimation 5 Bipartite Graph Networks with Set Transformer for Trigger Detection 5.1 Set Attention Blocks 5.2 Bipartite Aggregation 6 Experiment Results 6.1 Dataset and Experiment Settings 6.2 Transverse Momentum Estimation 6.3 Trigger Detection 7 Conclusions References Understanding Difficulty-Based Sample Weighting with a Universal Difficulty Measure 1 Introduction 2 Preliminaries 2.1 Description of Symbols 2.2 Definition of the Generalization Error 2.3 Conditions and Definitions 2.4 Experiment Setup 3 A Universal Difficulty Measure 3.1 Noise Factor 3.2 Imbalance Factor 3.3 Margin Factor 3.4 Uncertainty Factor 3.5 Discussion About Generalization Error 4 Role of Difficulty-Based Weighting 4.1 Effects on Optimization Dynamics 4.2 Effects on Generalization Performance 5 Discussion 6 Conclusion References Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks 1 Introduction 2 Related Work 3 Network Structure Altering 3.1 Connection-Level Actions 3.2 Neuron-Level Actions 4 Proposed Method 4.1 Selection of a New Sub-network 4.2 Addressing Class Ambiguities 4.3 Training 5 Experiments and Results 5.1 Results 5.2 Analysis 6 Conclusion References PrUE: Distilling Knowledge from Sparse Teacher Networks 1 Introduction 2 Related Work 3 Background 3.1 Preliminaries 3.2 Prediction Uncertainty 4 Prediction Uncertainty Enlargement 5 Experiments 5.1 The Effect of LS on Knowledge Distillation 5.2 Comparison with Other Distillation Methods 5.3 Comparison with Other Pruning Methods 5.4 Distillation on Large-Scale Datasets 6 Conclusion References Robust and Adversarial Machine Learning Fooling Partial Dependence via Data Poisoning 1 Introduction 2 Related Work 3 Partial Dependence 4 Fooling Partial Dependence via Data Poisoning 4.1 Attack Loss 4.2 Genetic-Based Algorithm 4.3 Gradient-Based Algorithm 5 Experiments 6 Limitations and Future Work 7 Conclusion and Impact References FROB: Few-Shot ROBust Model for Joint Classification and Out-of-Distribution Detection 1 Introduction 2 Proposed Methodology for Few-Shot OoD Detection 3 Related Work on Classification and OoD Detection 4 Evaluation and Results 4.1 Evaluation of FROB 4.2 Ablation Studies 5 Conclusion References PRoA: A Probabilistic Robustness Assessment Against Functional Perturbations 1 Introduction 2 Related Work 3 Preliminary 4 Verification of Probabilistic Robustness 4.1 Formulating Verification Problem 4.2 Adaptive Concentration Inequalities 4.3 Verification Algorithm 5 Experiments 5.1 Baseline Setting 5.2 Considered Functional Perturbations 5.3 Quantitative Results of Experiments on CIFAR-10 5.4 Comparing Probabilistic Robustness Across Models on ImageNet 6 Conclusion References Hypothesis Testing for Class-Conditional Label Noise 1 Introduction 2 Background 3 Hypothesis Tests Based on Anchor Points 3.1 A Hypothesis Test for Class-Conditional Label Noise 3.2 Multiple Anchor Points 3.3 Multiple Relaxed Anchors-Points 3.4 What if We have No Anchor Points? 3.5 Practical Considerations and Limitations 4 Related Work 5 Experiments 6 Conclusion and Future Work References On the Prediction Instability of Graph Neural Networks 1 Introduction 2 Preliminaries and Experimental Setup 3 Results 3.1 Overall Prediction Instability of GNNs 3.2 The Effect of Node Properties 3.3 The Effect of Model Design and Training Setup 3.4 Layer-Wise Model Introspection 4 Discussion 5 Related Work 6 Conclusion References Adversarially Robust Decision Tree Relabeling 1 Introduction 2 Background Information 2.1 Decision Tree Learning 2.2 Robust Decision Tree Learning 2.3 Minimum Vertex Covers and Robustness 2.4 Relabeling and Pruning Decision Trees 3 Robust Relabeling 3.1 Robust Relabeling as Splitting Criterion 3.2 Runtime Comparison 4 Improving Robustness 4.1 Decision Trees 4.2 Decision Tree Ensembles 4.3 Adversarial Pruning 4.4 Accuracy Robustness Trade-Off 5 Regularizing Decision Trees 5.1 Toy Datasets 5.2 Comparison with Cost Complexity Pruning 6 Conclusions References Calibrating Distance Metrics Under Uncertainty 1 Introduction 2 Background 2.1 Missing Data and Imputation 2.2 Metric Calibration 3 Model 3.1 A Kernel's Trick 3.2 Direct Calibration 3.3 Dykstra's Algorithm 3.4 Cyclic Calibration 3.5 Complexity Analysis 4 Evaluation 4.1 Settings 4.2 Noise Reduction on Distance Metrics 4.3 Distance Metrics from Incomplete Data 4.4 Classification on Incomplete Samples 4.5 Scalability 5 Conclusion References Defending Observation Attacks in Deep Reinforcement Learning via Detection and Denoising 1 Introduction 2 Background 2.1 Markov Decision Process 2.2 Observation Attack 2.3 Defense via Detection and Denoising 2.4 Online and Offline Sampling 3 Approach 3.1 Defense 3.2 Observation Attacks 4 Experiments 4.1 Rewards Under Attack w/wo Defense (Q1) 4.2 Non-adversarial Scenarios (Q2) and Comparison (Q3) 4.3 Detector and Denoiser (Q4) 4.4 Adaptive Attack (Q5) 5 Conclusion References Resisting Graph Adversarial Attack via Cooperative Homophilous Augmentation 1 Introduction 2 Preliminaries 2.1 Graph Convolutional Network 2.2 Graph Adversarial Attack 3 The Proposed Framework 3.1 Heterophily and Attack 3.2 Cooperative Homophilous Augmentation 3.3 Theoretical Guarantee 4 Experiment 4.1 Experimental Setup 4.2 Defense Performance Against Non-targeted Adversarial Attacks 4.3 Defense Performance Under Different Injected Nodes Ratio 4.4 Parameter Sensitivity on Eliminating Rate 5 Related Work 5.1 Adversarial Attacks on GNNs 5.2 Defenses on GNNs 6 Conclusion References Securing Cyber-Physical Systems: Physics-Enhanced Adversarial Learning for Autonomous Platoons 1 Introduction 2 Problem Definition 3 Attacker Model 3.1 Conventional Cyber-Physical Attacks 3.2 Adversarially-Masked Cyber-Physical Attacks 3.3 Attacker Capabilities 4 Physics-Enhanced Defense Approach 4.1 Case Study: Autonomous Vehicle Platoons 4.2 Data-Driven Anomaly Detector 4.3 Physical Consistency Checker 5 Experimental Results 5.1 Simulation Setup 5.2 Double-Insured Anomaly Detection (DAD) 5.3 Evaluation Setup 5.4 Attack Detection Results 5.5 Simulation Demonstration for the M-FDI Attack 6 Conclusions References MEAD: A Multi-Armed Approach for Evaluation of Adversarial Examples Detectors 1 Introduction 1.1 Summary of Contributions 2 Related Works 2.1 Considered Detection Methods 2.2 Considered Attack Mechanisms 3 Adversarial Examples and Novel Objectives 3.1 Generating Adversarial Examples 3.2 Three New Objective Functions 3.3 A Case Study: ACE vs. Gini Impurity 4 Evaluation with a Multi-armed Attacker 5 Experiments 5.1 Experimental Setting 5.2 Experimental Results 6 Summary and Concluding Remarks References Adversarial Mask: Real-World Universal Adversarial Attack on Face Recognition Models 1 Introduction 2 Background and Related Work 2.1 Adversarial Attacks 2.2 Face Recognition 3 Method 3.1 Mask Projection 3.2 Patch Optimization 4 Evaluation 4.1 Digital Attacks 4.2 Physical Attacks 5 Countermeasures 6 Conclusion References Generative Models TrafficFlowGAN: Physics-Informed Flow Based Generative Adversarial Network for Uncertainty Quantification 1 Introduction 2 Background and Related Work 2.1 Normalizing Flow 2.2 Generative Adversarial Network (GAN) 3 Framework of TrafficFlowGAN 3.1 Problem Statement 3.2 Overview of TrafficFlowGAN Structure 3.3 Convolutional Neural Network as the Discriminator 3.4 Conditional Flow as the Generator 3.5 Physics Regularization 3.6 Training of TrafficFlowGAN 4 Numerical Experiment: Learning Solutions of a Known Second-Order PDE 4.1 Numerical Data 4.2 Physics-Based Computational Graph 4.3 Experiment Setting 4.4 Results 5 Case Study: Traffic State Estimation 5.1 Dataset 5.2 Physics-Based Computational Graph 5.3 Baselines and Metrics 5.4 Results 6 Conclusions References STGEN: Deep Continuous-Time Spatiotemporal Graph Generation 1 Introduction 2 Related Works 3 Problem Setting 4 Generative Model for Spatiotemporal Graph 4.1 Overall Architecture 4.2 Spatiotemporal Walk Generator with Validity Constraints 4.3 Spatiotemporal Walk Discriminator 5 Experiment 5.1 Quantitative Performance 5.2 Case Study 5.3 Model Scalability 6 Conclusion References Direct Evolutionary Optimization of Variational Autoencoders with Binary Latents 1 Introduction and Related Work 2 Direct Variational Optimization 3 Numerical Experiments 4 Discussion References Scalable Adversarial Online Continual Learning 1 Introduction 2 Related Works 3 Problem Formulation 4 Adversarial Continual Learning (ACL) 5 Learning Policy of SCALE 5.1 Feature Transformation 5.2 Loss Function 5.3 Meta-training Strategy 5.4 Adversarial Training Strategy 6 Experiments 6.1 Datasets 6.2 Baselines 6.3 Implementation Notes 6.4 Numerical Results 6.5 Memory Analysis 6.6 Ablation Study 6.7 Execution Times 7 Sensitivity Analysis 8 Conclusion References Fine-Grained Bidirectional Attention-Based Generative Networks for Image-Text Matching 1 Introduction 2 Related Work 3 Proposed Method 3.1 Framework 3.2 Positional Embedding 3.3 Object Relational Reasoning 3.4 Bidirectional Attentional Generative Network 3.5 Global and Local Multi-modal Cross-Attention 3.6 Object Function 4 Experiments 4.1 Quantitative Analysis 4.2 Qualitative Analysis 4.3 GCN Parameter Analysis 4.4 Visualization 4.5 Ablation Experiments 5 Conclusion References Computer Vision Learnable Masked Tokens for Improved Transferability of Self-supervised Vision Transformers 1 Introduction 2 Vision Transformers with Learnable Masked Tokens 2.1 Background: Vision Transformers 2.2 Learning the Selection Function 2.3 Regularizations on Masked Tokens 3 Experiments 3.1 Configurations 3.2 In-domain Transfer Learning 3.3 Cross-Domain Transfer Learning 4 Related Works 5 Conclusions References Rethinking the Misalignment Problem in Dense Object Detection 1 Introduction 2 Related Work 3 Proposed Approach 3.1 SALT: Regression-Aware Points 3.2 SALT: Classification-Aware Points 3.3 SALT: Feature Alignment 3.4 Self-distillation 3.5 Loss Function 4 Experiments 4.1 Performance of SALT's Component Parts 4.2 The Selection of Spatial Disentanglement Strategies 4.3 Generality of SALT 4.4 Self-distillation Regression Loss 4.5 Evaluations for Task-Alignment of SALT-Net 4.6 Comparisons with State-of-the-Arts 5 Conclusion References No More Strided Convolutions or Pooling: A New CNN Building Block for Low-Resolution Images and Small Objects 1 Introduction 2 Preliminaries and Related Work 2.1 Small Object Detection 2.2 Low-Resolution Image Classification 3 A New Building Block: SPD-Conv 3.1 Space-to-depth (SPD) 3.2 Non-strided Convolution 4 How to Use SPD-Conv: Case Studies 4.1 Object Detection 4.2 Image Classification 5 Experiments 5.1 Object Detection 5.2 Image Classification 6 Conclusion References SAViR-T: Spatially Attentive Visual Reasoning with Transformers 1 Introduction 2 Related Works 2.1 Abstract Visual Reasoning 2.2 Transformer in Vision 3 Method 3.1 Raven's Progressive Matrices 3.2 Our Approach: SAViR-T 3.3 Training and Inference 4 Experiments 4.1 Experimental Settings 4.2 Performance Analysis 4.3 Ablation Study 5 Conclusions References A Scaling Law for Syn2real Transfer: How Much Is Your Pre-training Effective? 1 Introduction 2 Related Work 3 Scaling Laws for Pre-training and Fine-tuning 3.1 Induction of Scaling Law with Small Empirical Results 3.2 Theoretical Deduction of Scaling Law 3.3 Insights and Practical Values 4 Experiments 4.1 Settings 4.2 Scaling Law Universally Explains Downstream Performance for Various Task Combinations 4.3 Bigger Models Reduce the Transfer Gap 4.4 Scaling Law Can Extrapolate for More Pre-training Images 4.5 Data Complexity Affects both Pre-training Rate and Transfer Gap 5 Conclusion and Discussion 5.1 Implication of Complexity Results in Sect.4.5 5.2 Lessons to Transfer Learning and Synthetic-to-Real Generalization 5.3 Limitations References Submodular Meta Data Compiling for Meta Optimization*-12pt 1 Introduction 2 Related Work 2.1 Meta Optimization 2.2 Meta Data Compiling 2.3 Submodular Optimization 3 Methodology 3.1 Theoretical Analysis for Meta Data Construction 3.2 Details of the Four Selection Criteria 3.3 Submodular Optimization 4 Experiments 4.1 Evaluation on CIFAR10 and CIFAR100 4.2 Evaluation of Large Data Sets 4.3 Discussion 5 Conclusions References Supervised Contrastive Learning for Few-Shot Action Classification*-12pt 1 Introduction 2 Related Work 3 Methodology 3.1 Framework Overview 3.2 Supervised Contrastive Learning 3.3 Few-Shot Classification 3.4 Total Learning Objective 4 Experiments 4.1 Datasets and Settings 4.2 Main Results 4.3 Further Evaluations 5 Conclusions References A Novel Data Augmentation Technique for Out-of-Distribution Sample Detection Using Compounded Corruptions*-12pt 1 Introduction 2 Related Work 3 Proposed Approach 3.1 Problem Formulation 3.2 Synthetic OOD Data Generation 3.3 CnC Analysis via Polyhedral Decomposition of Input Space 3.4 Training Procedure 3.5 Inference 4 Dataset and Evaluation Methodology 5 Experiments and Results 5.1 Comparison with State-of-the-art 5.2 Other Benefits of CnC 5.3 Ablation Studies 6 Conclusions References Charge Own Job: Saliency Map and Visual Word Encoder for Image-Level Semantic Segmentation*-12pt 1 Introduction 2 Related Work 2.1 Weakly-Supervised Semantic Segmentation 2.2 Saliency Detection 3 Proposed Method 3.1 Motivation 3.2 Semantic Word Learning 3.3 Saliency Map Feedback 3.4 Jointly Learning of Pseudo Label Generation 4 Experiments 4.1 Experimental Setup 4.2 Ablation Study and Analysis 4.3 Comparison with State-of-the-Arts 5 Conclusion References Understanding Adversarial Robustness of Vision Transformers via Cauchy Problem 1 Introduction 2 Related Work 2.1 Vision Transformers and Its Variants 2.2 Robustness of Vision Transformer 2.3 Deep Neural Network via Dynamic Point of View 3 Preliminaries 4 Theoretically Analysis 4.1 Vision Transformers are Lipschitz 4.2 Model Adversarial Robustness as Cauchy Problem 5 Empirical Study 5.1 Configuration and Training Result 5.2 Study for Small Scale Models 5.3 Contribution of MSA to Robustness for Large Scale Models 6 Conclusion 7 Discussion and Limitation References Meta-learning, Neural Architecture Search Automatic Feature Engineering Through Monte Carlo Tree Search 1 Introduction 2 Related Work 3 Methodology 3.1 The Transformation Tree 3.2 The Selection Policy 3.3 The Expansion Policy 3.4 The mCAFE Algorithm 4 Evaluation 4.1 Performance of mCAFE 4.2 Ablation Study 4.3 Length of Feature Engineering Pipeline 4.4 Performances of mCAFE on Different Predictive Models 5 Conclusion and Future Work References MRF-UNets: Searching UNet with Markov Random Fields*-10pt 1 Introduction 2 Related Works 3 Preliminaries 3.1 Markov Random Field 3.2 Diverse M-Best Inference 3.3 AOWS 4 MRF-NAS 4.1 Diverse M-Best Loopy Inference 4.2 Differentiable Parameter Learning 5 Experiments 5.1 Datasets 5.2 Search Space 5.3 Implementation Details 5.4 Computational Cost 5.5 MRF-UNets Architecture 5.6 Main Results 6 Ablation Study 6.1 Exact vs. Approximate Loopy Inference 6.2 Diverse Solutions 6.3 Pairwise Formulation and Differentiable Parameter Learning 7 Conclusion References Adversarial Projections to Tackle Support-Query Shifts in Few-Shot Meta-Learning*-12pt 1 Introduction 2 Related Work 2.1 Cross-Domain Few-shot Learning (CDFSL) 2.2 Support-Query Shift in FSL 3 Methodology 3.1 Preliminaries 3.2 Adversarial Query Projection (AQP) 4 Experiments and Results 4.1 Implementation Details 4.2 Contributions to FewShiftBed 4.3 Evaluation of SQS+ 4.4 Evaluation of AQP 4.5 Ablations 4.6 Visual Analysis of AQP 5 Conclusion and Future Directions References Discovering Wiring Patterns Influencing Neural Network Performance*-12pt 1 Introduction 2 Related Work 3 From a Graph to a Neural Network 4 The Space of Random Graphs and DAGs 4.1 Direct Construction of Random DAGs 5 Results 5.1 The Inadequacy of Classical Graph Characteristics 5.2 The Worst Networks 5.3 The Best Networks 6 Impact on Architecture Design 6.1 Long- vs. Short-range Connections 6.2 Influence of Bottlenecks 6.3 CIFAR-10 Versus CIFAR-100 Consistency 7 Conclusions and Outlook References Context Abstraction to Improve Decentralized Machine Learning in Structured Sensing Environments 1 Introduction 2 Background and Motivation 2.1 IoT Deployments 2.2 Decentralized Machine Learning 2.3 IoT Deployments and Impact of the Context 2.4 Relativity of Viewpoints in Structured Sensing Environments 3 FedAbstract Algorithm 3.1 Learning Group-Invariant and Position-Specific Representations 3.2 Relative Geometry for Data Generators 3.3 Conciliation Process 4 Experiments and Results 4.1 Performance Comparison 4.2 Ablation Study 5 Conclusion and Future Work References Efficient Automated Deep Learning for Time Series Forecasting*-15pt 1 Introduction 2 Related Work 2.1 Deep Learning Based Forecasting 2.2 Automated Deep Learning (AutoDL) 2.3 AutoML for Time Series Forecasting 3 AutoPyTorch Forecasting 3.1 Problem Definition 3.2 Evaluating Forecasting Pipelines 3.3 Forecasting Pipeline Configuration Space 3.4 Hyperparameter Optimization 3.5 Proxy-Evaluation on Many Time Series 4 Experiments 4.1 Time Series Forecasting 4.2 Hyperparameter Importance 4.3 Ablation Study 5 Conclusion and Future Work References Author Index