Advances of Multisensory Integration in the Brain (Advances in Experimental Medicine and Biology Book 1437)

This book presents the latest research on multisensory brain function. Namely, the mechanisms by which the brain processes and integrates information from multiple sensory modalities. Its contents cover a broad range of topics, including optimal integration, cross-modal interactions, calibration, and causal inference -- with an emphasis on their neuronal underpinnings. By bringing together efforts from different laboratories around the world we aim to collaboratively shed light on these fundamental brain processes, that underlie perception, cognition, and behavior in a complex multisensory world, and to spur innovation of brain-inspired technologies Preface Contents 1: Decentralized Neural Circuits of Multisensory Information Integration in the Brain 1.1 Introduction 1.1.1 The Bayesian Model of Multisensory Integration 1.1.2 Towards the Neural Architecture of Multisensory Integration 1.2 A Decentralized Network of Multisensory Integration 1.2.1 The Network Model 1.2.2 Recurrent Connections in the Network 1.2.3 Feedforward Inputs from Unisensory Brain Areas 1.3 Neuronal Responses in the Decentralized Network 1.4 Bayesian Inference in the Decentralized Network 1.4.1 Feedforward Inputs Convey the Likelihood Distribution 1.4.2 The Network Dynamics on the Stimulus Subspace 1.4.3 Approximate Inference of the Posterior by Sampling 1.4.4 Sampling-Based Inference in the Stimulus Subspace in Decentralized Network 1.4.5 The Stimulus Prior Is Stored in Reciprocal Connections Between Networks 1.5 A Large-Scale Decentralized Network Is Robust to Local Failure Appendix The Probabilistic Model of Multisensory Integration Neural Dynamics of the Decentralized Network Unisensory Neurons’ Response Convey the Likelihood Function Theoretical Analysis of the Network Dynamics on the Stimulus Subspace References 2: From Multisensory Integration to Multisensory Decision-Making 2.1 Computational Modeling 2.2 Physiological Studies 2.2.1 Brain Regions Associated with Multisensory Decision-Making 2.2.2 Modality and Category-Free Coding 2.2.3 Modality-Dependent Dynamics of Decision Signals 2.2.4 Causality Issue References 3: More Than the Sum of Its Parts: Visual–Tactile Integration in the Behaving Rat 3.1 Introduction 3.2 Evolution of Multisensory Perception 3.3 Models of Multisensory Cue Combination 3.3.1 Linear Models for Maximum Reliability 3.3.2 Bayesian Cue Integration 3.3.3 Bayesian Modeling of Multisensory Behavioral Experiments 3.4 Rodent Models for the Study of Perception 3.4.1 Rat Tactile Sensory System 3.4.2 Rat Vision 3.5 Visual–Tactile Orientation Categorization Task 3.6 Convergence of Sensory Pathways in the Posterior Parietal Cortex References 4: Multisensory Integration and Causal Inference in Typical and Atypical Populations 4.1 The Challenges of Multisensory Processing 4.2 Integrating Signals that Share a Source: Forced Fusion 4.2.1 Bayesian Modelling of Forced-Fusion Integration 4.2.2 Forced-Fusion Integration in Healthy Young Adults 4.2.3 Forced-Fusion Integration in Other Populations 4.2.3.1 Children 4.2.3.2 Older Adults 4.2.3.3 Atypical Populations 4.2.4 Neural Mechanisms of Reliability-Weighted Integration 4.3 Processing Signals from Multiple Sources: Causal Inference 4.3.1 Bayesian Modelling of Multisensory Causal Inference 4.3.2 Bayesian Causal Inference in Healthy Young Adults 4.3.3 Bayesian Causal Inference in Other Populations 4.3.3.1 Children 4.3.3.2 Older Adults 4.3.3.3 Atypical Populations 4.3.4 Neural Mechanisms of Bayesian Causal Inference 4.4 Conclusion References 5: Multisensory Integration in Body Representation 5.1 Temporal and Spatial Constraints on Multisensory Integration 5.2 Prior Knowledge of the Body Influences Multisensory Integration 5.3 A Bayesian Framework of Multisensory Integration 5.4 Neurophysiological Evidence of Body Representation 5.4.1 Neuronal Basis of Multisensory Integration of Body-Related Signals 5.4.2 Neuronal Representation of One’s Own Body 5.4.3 Electrophysiological Evidence of Causal Inference in Body Representation References 6: Crossmodal Associations and Working Memory in the Brain 6.1 Crossmodal Associations and Working Memory 6.2 Where Are Crossmodal Associations and Working Memory in the Brain? 6.3 When Are Crossmodal Associations Stored in Working Memory? 6.4 How Are Crossmodal Associations and Working Memory Processed? 6.5 Causal Evidence in Neural Mechanisms 6.6 Concluding Remarks References 7: Synesthetic Correspondence: An Overview 7.1 Introduction 7.2 Synesthetic Correspondence and Cross-modal Integration 7.3 Sound Symbolism as a Form of Synesthetic Correspondence 7.4 Neurocognitive Mechanism of Synesthetic Correspondence 7.5 Cognitive Benefits of Synesthetic Associations and the Role of Training (Experience) 7.6 Synesthetic Correspondence in Sensory Branding 7.7 Sensory Linguistics Perspective: Plurality or Ambiguous Meanings 7.8 Applications: Synesthesia Device 7.9 Summary: New Directions for Synesthetic Correspondences 7.9.1 Synesthetic Correspondence in Embodied Cognition and Human-Machine (AI) Interaction 7.9.2 Knowledge (Concepts) Representation 7.9.3 Synesthetic Correspondence, Metacognition, and Multisensory Integration 7.9.4 Cross-modal Correspondence in Virtual Reality 7.10 Concluding Remarks References 8: Neural Oscillations and Multisensory Processing 8.1 Neural Oscillations and Sensory Processing 8.2 Cross-Frequency Coupling and Sensory Processing 8.3 Neural Oscillations and Multisensory Processing 8.4 Neural Oscillations and Sensory Selection 8.5 Summary References 9: Multisensory Calibration: A Variety of Slow and Fast Brain Processes Throughout the Lifespan 9.1 Introduction 9.2 Multisensory Calibration Across the Lifespan 9.3 Unsupervised Multisensory Calibration 9.4 Supervised Multisensory Calibration 9.5 Rapid Calibration: Adaptation and Serial Dependence 9.6 Neural Correlates of Multisensory Calibration 9.7 Concluding Remarks References 10: The Development of Multisensory Integration at the Neuronal Level 10.1 Introduction 10.2 Development of Multisensory Integration in the Superior Colliculus (SC) 10.3 The Influence of Sensory Deprivation on Multisensory Integration 10.4 The Role of Multisensory Experience in the Development of Multisensory Integration 10.5 Sensory Loss and Cross-Modal Plasticity 10.6 The Influence of Perceptual Association Learning on Cortical Cross-Modal Plasticity References