Multimodal learning for clinical decision support and clinical image-based procedures : 10th international workshop, ML-CDS 2020, and 9th international workshop, CLIP 2020, held in conjunction with MICCAI 2020, Lima, Peru, October 4-8, 2020 : proceedings

This book constitutes the refereed joint proceedings of the 10th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2020, and the 9th International Workshop on Clinical Image-Based Procedures, CLIP 2020, held in conjunction with the 23rd International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2020, in Lima, Peru, in October 2020. The workshops were held virtually due to the COVID-19 pandemic. The 4 full papers presented at ML-CDS 2020 and the 9 full papers presented at CLIP 2020 were carefully reviewed and selected from numerous submissions to ML-CDS and 10 submissions to CLIP. The ML-CDS papers discuss machine learning on multimodal data sets for clinical decision support and treatment planning. The CLIP workshops provides a forum for work centered on specific clinical applications, including techniques and procedures based on comprehensive clinical image and other data. Additional Workshop Editors 6 Preface ML-CDS 2020 7 Organization 8 Preface CLIP 2020 9 Organization 10 Contents 11 CLIP 2020 13 Optimal Targeting Visualizations for Surgical Navigation of Iliosacral Screws 14 1 Introduction 14 2 Methods 15 3 Results 18 4 Discussion and Conclusion 20 References 22 Prediction of Type II Diabetes Onset with Computed Tomography and Electronic Medical Records 24 1 Introduction 24 2 Method 26 2.1 Data of T2DM Studies 26 2.2 Abdominal Segmentations 27 2.3 EMR Feature Extraction 28 3 Experiments 29 3.1 Experimental Design 29 3.2 Implementation Details and Metric 29 3.3 Results and Analyses 30 4 Discussion and Conclusion 31 References 32 A Radiomics-Based Machine Learning Approach to Assess Collateral Circulation in Ischemic Stroke on Non-contrast Computed Tomography 35 1 Introduction 35 2 Materials and Methods 37 2.1 Scanning Protocols 37 2.2 Ground Truth Labels 38 2.3 Mapping of ASPECTS Regions 38 2.4 Pre-processing 38 2.5 Image Features 39 2.6 Classification of Collaterals 40 3 Results 41 4 Discussion 41 References 42 Image-Based Subthalamic Nucleus Segmentation for Deep Brain Surgery with Electrophysiology Aided Refinement 45 1 Introduction 45 2 Methods 46 2.1 Data 46 2.2 MER Acquisition and Preprocessing 47 2.3 MRI Data Processing 47 2.4 Active Contours Fitting 47 2.5 MER-based Fitting 49 2.6 Evaluation Procedure 49 3 Results and Discussions 49 3.1 STN Segmentation 50 3.2 MER-based Fitting 51 4 Conclusion 52 References 53 3D Slicer Craniomaxillofacial Modules Support Patient-Specific Decision-Making for Personalized Healthcare in Dental Research 55 1 Introduction 55 2 Methods 56 3 Results and User Studies Applications 57 4 Conclusion 62 References 63 Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision 65 1 Introduction 65 2 Prior Work 66 3 Method 67 3.1 Dataset 67 3.2 Variational Autoencoder 67 3.3 Training 68 3.4 Model Evaluation 68 4 Results 70 5 Conclusion and Future Work 72 References 73 Single-Shot Deep Volumetric Regression for Mobile Medical Augmented Reality 75 1 Introduction 75 2 Materials and Methods 76 2.1 Data Acquisition and Preprocessing 77 2.2 Android Application 77 2.3 Server Backend 79 3 Results 80 3.1 Quantitative Evaluation 80 3.2 Qualitative Evaluation 80 4 Discussion 81 5 Conclusion 83 References 84 A Baseline Approach for AutoImplant: The MICCAI 2020 Cranial Implant Design Challenge 86 1 Introduction 86 2 Dataset 88 3 Method 89 4 Experiments and Results 90 5 Conclusion and Future Improvement 94 References 94 Adversarial Prediction of Radiotherapy Treatment Machine Parameters 96 1 Introduction 96 2 Methods 97 2.1 Patient Data Preparation and Treatment Planning 97 2.2 Data Reformatting for Supervised Learning 97 2.3 Deep Learning Network and Training 99 3 Results 101 3.1 Plan Quality Overview via Dose Volume Histograms 101 3.2 Dosimetric Properties of the Inferred Treatment Plans 102 3.3 Predicted-Plan v. Predicted-Dose Results 103 4 Summary and Discussion 103 References 104 ML-CDS 2020 106 Soft Tissue Sarcoma Co-segmentation in Combined MRI and PET/CT Data 107 1 Introduction 107 2 Method 109 3 Experimental Setup 111 3.1 Dataset and Pre-processing 111 3.2 Network Training 111 3.3 Evaluation Measures 112 4 Results and Discussion 112 5 Conclusion 114 References 114 Towards Automated Diagnosis with Attentive Multi-modal Learning Using Electronic Health Records and Chest X-Rays 116 1 Introduction 116 1.1 Related Work 117 2 Proposed Approach 118 3 Data and Experimental Setup 120 3.1 Tasks 120 3.2 Experimental Setup 121 4 Results 121 5 Conclusions and Future Work 123 References 123 LUCAS: LUng CAncer Screening with Multimodal Biomarkers 125 1 Introduction 125 2 Lung Cancer Screening (LUCAS) Dataset 128 2.1 Visual Information 128 2.2 Biomarkers 128 2.3 Task 129 3 Baseline Approach 129 3.1 Image Pre-processing 129 3.2 Method 129 3.3 Training Details 130 4 Results 131 5 Conclusions 132 References 133 Automatic Breast Lesion Classification by Joint Neural Analysis of Mammography and Ultrasound 135 1 Introduction 135 2 Related Work 136 3 Method 137 3.1 Dataset 137 3.2 Model Architecture 138 3.3 Implementation Details 138 4 Experiments 140 4.1 Leave-One-Out 140 4.2 Reader Study 141 4.3 Model Insight 142 5 Conclusion 143 References 143 Author Index 146 Front Matter ....Pages i-xii Front Matter ....Pages 1-1 Optimal Targeting Visualizations for Surgical Navigation of Iliosacral Screws (Prashant U. Pandey, Pierre Guy, Kelly A. Lefaivre, Antony J. Hodgson)....Pages 3-12 Prediction of Type II Diabetes Onset with Computed Tomography and Electronic Medical Records (Yucheng Tang, Riqiang Gao, Ho Hin Lee, Quinn Stanton Wells, Ashley Spann, James G. Terry et al.)....Pages 13-23 A Radiomics-Based Machine Learning Approach to Assess Collateral Circulation in Ischemic Stroke on Non-contrast Computed Tomography (Mumu Aktar, Yiming Xiao, Donatella Tampieri, Hassan Rivaz, Marta Kersten-Oertel)....Pages 24-33 Image-Based Subthalamic Nucleus Segmentation for Deep Brain Surgery with Electrophysiology Aided Refinement (Igor Varga, Eduard Bakstein, Greydon Gilmore, Daniel Novak)....Pages 34-43 3D Slicer Craniomaxillofacial Modules Support Patient-Specific Decision-Making for Personalized Healthcare in Dental Research (Jonas Bianchi, Beatriz Paniagua, Antonio Carlos De Oliveira Ruellas, Jean-Christophe Fillion-Robin, Juan C. Prietro, João Roberto Gonçalves et al.)....Pages 44-53 Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision (David Z. Li, Masaru Ishii, Russell H. Taylor, Gregory D. Hager, Ayushi Sinha)....Pages 54-63 Single-Shot Deep Volumetric Regression for Mobile Medical Augmented Reality (Florian Karner, Christina Gsaxner, Antonio Pepe, Jianning Li, Philipp Fleck, Clemens Arth et al.)....Pages 64-74 A Baseline Approach for AutoImplant: The MICCAI 2020 Cranial Implant Design Challenge (Jianning Li, Antonio Pepe, Christina Gsaxner, Gord von Campe, Jan Egger)....Pages 75-84 Adversarial Prediction of Radiotherapy Treatment Machine Parameters (Lyndon Hibbard)....Pages 85-94 Front Matter ....Pages 95-95 Soft Tissue Sarcoma Co-segmentation in Combined MRI and PET/CT Data (Theresa Neubauer, Maria Wimmer, Astrid Berg, David Major, Dimitrios Lenis, Thomas Beyer et al.)....Pages 97-105 Towards Automated Diagnosis with Attentive Multi-modal Learning Using Electronic Health Records and Chest X-Rays (Tom van Sonsbeek, Marcel Worring)....Pages 106-114 LUCAS: LUng CAncer Screening with Multimodal Biomarkers (Laura Daza, Angela Castillo, María Escobar, Sergio Valencia, Bibiana Pinzón, Pablo Arbeláez)....Pages 115-124 Automatic Breast Lesion Classification by Joint Neural Analysis of Mammography and Ultrasound (Gavriel Habib, Nahum Kiryati, Miri Sklair-Levy, Anat Shalmon, Osnat Halshtok Neiman, Renata Faermann Weidenfeld et al.)....Pages 125-135 Back Matter ....Pages 137-138