Advances in Intelligent Data Analysis XIX: 19th International Symposium on Intelligent Data Analysis, IDA 2021, Porto, Portugal, April 26–28, 2021, ... (Lecture Notes in Computer Science, 12695)

This book constitutes the proceedings of the 19th International Symposium on Intelligent Data Analysis, IDA 2021, which was planned to take place in Porto, Portugal. Due to the COVID-19 pandemic the conference was held online during April 26-28, 2021. The 35 papers included in this book were carefully reviewed and selected from 113 submissions. The papers were organized in topical sections named: modeling with neural networks; modeling with statistical learning; modeling language and graphs; and modeling special data formats. Preface Organization Contents Modeling with Neural Networks Hyperspherical Weight Uncertainty in Neural Networks 1 Introduction 2 Background: On Gaussian Distributions 3 Hypersphere Bayesian Neural Networks 4 Results 4.1 Non-linear Regression 4.2 Image Classification 4.3 Measuring Uncertainty 4.4 Active Learning Using Uncertainty Quantification 4.5 Variational Auto-encoders 5 Conclusion References Partially Monotonic Learning for Neural Networks 1 Introduction 2 Related Work 3 Monotonicity 4 Partially Monotonic Learning 4.1 Loss Function 5 Evaluation 5.1 Datasets 5.2 Methodology 5.3 Monotonic Features Extraction 5.4 Models 5.5 Monotonicity Analysis 6 Conclusion and Future Work References Multiple-manifold Generation with an Ensemble GAN and Learned Noise Prior 1 Introduction 2 Related Work 3 Model 4 Experiments 4.1 Disconnected Manifolds 4.2 CelebA+Photo 4.3 Complex-But-Connected Image Dataset 4.4 CIFAR 5 Discussion References Simple, Efficient and Convenient Decentralized Multi-task Learning for Neural Networks 1 Introduction 2 The Method 2.1 Intuition 2.2 Description 3 Theoretical Analysis 4 Experiments 4.1 Setting 4.2 Results 5 Related Work 6 Conclusion References Deep Hybrid Neural Networks with Improved Weighted Word Embeddings for Sentiment Analysis 1 Introduction 2 Related Work 2.1 Sentiment Analysis 2.2 Vector Representation 3 Proposed Model 3.1 Embedding Layer 3.2 Convolution Layer 3.3 Max-Pooling and Dropout Layer 3.4 LSTM Layer 3.5 Fully-Connected Layer 3.6 Output Layer 4 Experiments and Results 4.1 Dataset Description 4.2 Parameters 4.3 Evaluation Metrics 4.4 Results and Discussion 5 Conclusion References Explaining Neural Networks by Decoding Layer Activations 1 Introduction 2 Method and Architecture 3 Theoretical Motivation of ClaDec 4 Assessing Interpretability and Fidelity 5 Evaluation 5.1 Qualitative Evaluation 5.2 Quantitative Evaluation 6 Related Work 7 Conclusions References Analogical Embedding for Analogy-Based Learning to Rank 1 Introduction 2 Analogy-Based Learning to Rank 3 Related Work 4 Analogical Embedding 4.1 Training the Embedding Network 4.2 Constructing Training Examples 5 Experiments 5.1 Data and Experimental Setup 5.2 Case Study 1: Analysing the Embedding Space 5.3 Case Study 2: Performance of able2rank 6 Conclusion References HORUS-NER: A Multimodal Named Entity Recognition Framework for Noisy Data 1 Introduction 2 Methodology and Features 3 Experimental Setup 4 Results and Discussion 5 Related Work 6 Conclusion References Modeling with Statistical Learning Incremental Search Space Construction for Machine Learning Pipeline Synthesis 1 Introduction 2 Preliminary and Related Work 3 DSWIZARD Methodology 3.1 Incremental Pipeline Structure Search 3.2 Hyperparameter Optimization 3.3 Meta-Learning 4 Experiments 4.1 Experiment Setup 4.2 Experiment Results 5 Conclusion References Adversarial Vulnerability of Active Transfer Learning 1 Introduction 2 Related Work 3 Attacking Active Transfer Learning 3.1 Threat Model 3.2 Feature Collision Attack 4 Implementation and Results 4.1 Active Transfer Learner Setup 4.2 Feature Collision Results 4.3 Impact on the Model 4.4 Hyper Parameters and Runtime 4.5 Adversarial Retraining Defense 5 Conclusion and Future Work References Revisiting Non-specific Syndromic Surveillance 1 Introduction 2 Non-specific Syndromic Surveillance 2.1 Problem Definition 2.2 Evaluation 3 Machine Learning Algorithms 3.1 Data Mining Surveillance System (DMSS) 3.2 What Is Strange About Recent Events? (WSARE) 3.3 Eigenevent 3.4 Anomaly Detection Algorithms 4 Basic Statistical Approaches 5 Experiments and Results 5.1 Evaluation Setup 5.2 Preliminary Evaluation 5.3 Results 6 Conclusion References Gradient Ascent for Best Response Regression 1 Introduction 2 Best Response Regression 2.1 Shortcomings of the Approach by Ben-Porat and Tennenholtz 3 Notation 4 Gradient Ascent Approach 5 Experiments 6 Conclusions References Intelligent Structural Damage Detection: A Federated Learning Approach 1 Introduction 2 Background 2.1 Autoencoder Deep Neural Network 3 Federated Learning Augmented with Tensor Data Fusion for SHM 3.1 Data Structure 3.2 Problem Formulation in Federated Learning 3.3 Tensor Data Fusion 3.4 The Client-Server Learning Phase 4 Related Work 5 Experimental Results 5.1 Data Collection 5.2 Results and Discussions 6 Conclusions References Composite Surrogate for Likelihood-Free Bayesian Optimisation in High-Dimensional Settings of Activity-Based Transportation Models 1 Introduction 2 Materials and Methods 2.1 Preday ABM 2.2 Bayesian Optimisation for Likelihood-Free Inference 2.3 Limitations of BOLFI for Calibrating Preday ABM 3 BOLFI with Composite Surrogate Model 4 Results 5 Summary and Conclusions References Active Selection of Classification Features 1 Introduction 2 Related Work 3 Utility-Based Active Selection of Classification Features 3.1 Unsupervised, Imputation Variance-Based Variant (U-ASCF) 3.2 Supervised, Probabilistic Selection Variant (S-ASCF) 4 Experimental Results 4.1 Comparative Results 4.2 Case Study 5 Conclusion References Feature Selection for Hierarchical Multi-label Classification 1 Introduction 2 Feature Selection 2.1 ReliefF 2.2 Information Gain 3 Related Work 4 Applying Feature Selection in HMC 4.1 Binary Relevance 4.2 Label Powerset 4.3 Our Proposal 5 Methodology 5.1 Datasets 5.2 Base Classifier 5.3 Evaluation Measures 6 Experiments and Discussion 7 Conclusion and Future Work References Bandit Algorithm for both Unknown Best Position and Best Item Display on Web Pages 1 Introduction 2 Related Work 3 Recommendation Setting 4 PB-MHB Algorithm 4.1 Sampling w.r.t. the Posterior Distribution 4.2 Overall Complexity 5 Experiments 5.1 Datasets 5.2 Competitors 5.3 Results 6 Conclusion References Performance Prediction for Hardware-Software Configurations: A Case Study for Video Games 1 Introduction 2 Learning Problem 3 Learning Model 3.1 Learning from Imprecise Observations 3.2 Enforcing Monotonicity Using a Penalty Term 3.3 Combined Loss 4 Case Study: Predicting FPS in Video Games 4.1 Dataset 4.2 Modeling Imprecise Observations 4.3 Experimental Design 4.4 Results 5 Related Work 6 Conclusion References AVATAR—Automated Feature Wrangling for Machine Learning 1 Introduction 2 Related Work 3 Data Wrangling for Machine Learning 3.1 Problem Statement 3.2 A Language for Feature Wrangling 3.3 Generating Arguments 4 Machine Learning for Feature Wrangling 4.1 Prune 4.2 Select 4.3 Evaluate 4.4 Wrangle 5 Evaluation 5.1 Wrangling New Features 5.2 Comparison with Humans 6 Conclusion and Future Work References Modeling Language and Graphs Semantically Enriching Embeddings of Highly Inflectable Verbs for Improving Intent Detection in a Romanian Home Assistant Scenario 1 Introduction 2 Related Work 3 Home Assistant Scenario and Challenges 4 Proposed Solution 5 Empirical Evaluations 5.1 Experimental Setup 5.2 Results and Discussions 6 Conclusions, Limitations, and Further Work Appendix A Confusion matrices and histograms References BoneBert: A BERT-based Automated Information Extraction System of Radiology Reports for Bone Fracture Detection and Diagnosis 1 Introduction 2 Related Works 2.1 Rule-Based Approaches 2.2 Machine Learning Approaches 2.3 Hybrid Approaches 3 Methodology 3.1 Dataset 3.2 Information Extraction 3.3 Training and Evaluation 4 Experiments 4.1 Assertion Classification 4.2 Named Entity Recognition 5 Discussion 6 Conclusion References Linking the Dynamics of User Stance to the Structure of Online Discussions 1 Introduction 2 Related Work 3 The Dynamics of User Stance and Dataset 4 Forecast User Stance Dynamics 4.1 A Supervised Machine Learning Problem 4.2 Predictive Features 4.3 Learning Stance in Twitter 4.4 Predictive Setup 5 Results 6 Conclusion References Unsupervised Methods for the Study of Transformer Embeddings 1 Introduction 2 Related Work 3 Unsupervised Methods for Layer Analysis 3.1 Matrix and Vector Representation of Layers 3.2 Measuring the Correlations Between Layers 3.3 Clustering Layers 3.4 Interpreting Layers 4 Experiments 4.1 Datasets and Models Used 4.2 Investigating the Correlations Between Layers 4.3 Identifying Clusters of Layers 4.4 Qualitative Interpretation 4.5 Quantitative Interpretation Using Dimension Reduction 4.6 Results Validation Using a Clustering Performance Metric 5 Conclusion References A Framework for Authorial Clustering of Shorter Texts in Latent Semantic Spaces 1 Introduction 2 Related Work 3 Our Approach 3.1 Building a Dense Feature Space with Non-parametric Topic Modeling 3.2 Unsupervised Authorial Clustering Variant 3.3 Semi-supervised Authorial Clustering Variant 4 Experimental Design 4.1 The Dataset 4.2 Configurations and Baselines 4.3 Results 5 Discussion and Conclusion References DeepGG: A Deep Graph Generator 1 Introduction 2 Related Work 3 Graph Construction Sequences 4 Deep Graph Generator 5 Experiments 5.1 Findings in Our Experiments 6 Conclusion References SINr: Fast Computing of Sparse Interpretable Node Representations is not a Sin! 1 Introduction 2 SINr: Algorithmic Framework 2.1 First Step: Community Detection Algorithms 2.2 Second Step: Node Predominance and Node Recall 3 Experimental Setup 3.1 Datasets 3.2 Extracting Word Co-occurrence Networks for Textual Corpora 3.3 State-of-the-Art Algorithms 4 Experiments and Results 4.1 SINr is Fast: Complexity and Runtime 4.2 Graph Embedding: Link Prediction 4.3 Word Embedding: Similarity 5 Conclusion References Detection of Contextual Anomalies in Attributed Graphs 1 Introduction 2 Related Work 2.1 Anomaly Detection with Vector Data 2.2 Anomaly Detection with Graph Data 2.3 Machine Learning and Graph Mining 3 Problem Definition 4 Our Method: CoBaGAD 5 Experiments 5.1 Dataset Generation and Description 5.2 Experimental Setup 6 Results 7 Conclusion References Ising-Based Louvain Method: Clustering Large Graphs with Specialized Hardware 1 Introduction 1.1 Related Work 2 Methods 2.1 Modularity Maximization 2.2 QUBO Models 2.3 Solving Community Detection on the Fujitsu Digital Annealer 2.4 Baseline Method: Louvain Algorithm 2.5 Ising-Louvain Method 2.6 Formulating Ising-Louvain 2.7 Implementation Details of Ising-Louvain 3 Results and Discussion 3.1 Experiments 3.2 Results 4 Conclusion References Modeling Special Data Formats Reducing Negative Impact of Noise in Boolean Matrix Factorization with Association Rules 1 Introduction 2 Preliminaries and Notation 2.1 Formal Concept Analysis 2.2 Association Rules 2.3 Boolean Matrix Factorization 3 Algorithm 3.1 New Algorithm: Grass 4 Evaluation 4.1 Comparison with Competitors 4.2 Ability to Handle Noise 5 Discussion References Z-Hist: A Temporal Abstraction of Multivariate Histogram Snapshots 1 Introduction 2 Background and Definitions 3 The Z-Hist Framework 3.1 Step I: Computation of the Mean Histogram Distribution 3.2 Step II: Calculation of Histogram Distances and Locations 3.3 Step III: Discretization of Multivariate Histogram Snapshots 3.4 Frequent Arrangement Mining and Disproportionality Analysis 4 Experiments 5 Conclusion References Muppets: Multipurpose Table Segmentation 1 Introduction 2 Table Segmentation 3 The Muppets framework 3.1 Constraints 3.2 Score 3.3 Algorithm 4 Evaluation 4.1 Single Column Type Detection 4.2 Semantic Error Detection 4.3 Table and Header Detection 5 Related Work 6 Conclusion and Future Work References SpLyCI: Integrating Spreadsheets by Recognising and Solving Layout Constraints 1 Introduction 2 Problem 3 Method 4 Experiments 5 Related Work 6 Conclusions and Future Work References RTL: A Robust Time Series Labeling Algorithm 1 Introduction 2 Related Work 3 Requirements for a Robust Time Series Labeling Method 4 Robust Time Series Labeling 4.1 Variable-Length Motifs 4.2 Representative Motifs 4.3 Motif Selection and Stopping Criteria 5 Results 6 Discussion 7 Conclusion and Future Research References The Compromise of Data Privacy in Predictive Performance 1 Introduction 2 Related Work 3 Problem Formulation and Approach 3.1 Handling the De-identification 3.2 Assessing Re-identification Risk 3.3 Data Utility 4 Experimental Methodology 4.1 Data Preparation 4.2 Privacy-Preservation 4.3 Classification Algorithms 4.4 Evaluation Measures 4.5 Results Analysis 5 Conclusion References Efficient Privacy Preserving Distributed K-Means for Non-IID Data 1 Introduction 2 Related Work 2.1 K-Means 2.2 Privacy Preserving Distributed K-Means 3 Efficient Privacy Preserving Distributed K-Means for Non-IID Data 3.1 Description 3.2 Evaluation 4 Conclusion References Author Index