Man-Machine Speech Communication: 18th National Conference, NCMMSC 2023, Suzhou, China, December 8–10, 2023, Proceedings (Communications in Computer and Information Science, 2006)

This book constitutes the refereed proceedings of the 18th National Conference on Man-Machine Speech Communication, NCMMSC 2023, held in Suzhou, China, during December 8–11, 2023.The 20 full papers and 11 short papers included in this book were carefully reviewed and selected from 117 submissions. They deal with topics such as speech recognition, synthesis, enhancement and coding, audio/music/singing synthesis, avatar, speaker recognition and verification, human–computer dialogue systems, large language models as well as phonetic and linguistic topics such as speech prosody analysis, pathological speech analysis, experimental phonetics, acoustic scene classification. Preface Organization Contents Ultra-Low Complexity Residue Echo and Noise Suppression Based on Recurrent Neural Network 1 Introduction 2 Problem Formulation 3 Proposed Method 3.1 Network Topologies 3.2 Loss Functions 4 Experiments 4.1 Complexity 4.2 Results 5 Conclusions References Semi-End-to-End Nested Named Entity Recognition from Speech 1 Introduction 2 Related Work 2.1 NER 2.2 NER from Speech 3 Methodology 3.1 NE Head Annotation 3.2 EHA-ASR 3.3 SpanNER 4 Experiments 4.1 Experimental Setup 4.2 Annotation Results 4.3 ASR Results 4.4 NER Results 4.5 Case Study 4.6 Results of Different NE Categories 5 Conclusion References A Lightweight Music Source Separation Model with Graph Convolution Network 1 Introduction 2 Related Work 2.1 Transformer-Based Methods 2.2 Graph Convolutional Network 3 Proposed Method 3.1 Encoder/Decoder 3.2 Graph Convolution Network Attention 3.3 Mask Estimator 3.4 Loss Function 4 Experiments 4.1 Dataset 4.2 Experimental Setup 5 Results and Analysis 5.1 Ablation Study 5.2 Comparison with Other MSS Models 6 Conclusion References Joint Time-Domain and Frequency-Domain Progressive Learning for Single-Channel Speech Enhancement and Recognition 1 Introduction 2 Progressive Learning with Joint Time Domain and Frequency Domain 2.1 Problem Formulation 2.2 Joint Time and Frequency Domain Progressive Learning 2.3 Multi-target Loss 3 Experiments and Analysis 3.1 Data Corpus 3.2 Implementation Details 3.3 Evaluation Metrics 3.4 Results and Analysis 4 Conclusion References A Study on Domain Adaptation for Audio-Visual Speech Enhancement 1 Introduction 2 Proposed System 2.1 Employed Model 2.2 Proposed Multi-Model Mixture Pseudo-Label Domain Adaptation (MMMP-DA) Method 3 Experiments 3.1 Datasets 3.2 Experimental Settings 3.3 Results of the MEASE, MTMEASE and PLMEASE in Matched Scenario 3.4 Performance Analysis of the Proposed MMMP-DA Method 3.5 Objective Results on AVSE Challenge 2023 4 Conclusions References APNet2: High-Quality and High-Efficiency Neural Vocoder with Direct Prediction of Amplitude and Phase Spectra 1 Introduction 2 Related Work 2.1 iSTFTNet 2.2 Vocos 2.3 APNet 3 Proposed Method 3.1 Model Structure 3.2 Training Criteria 4 Experiments 4.1 Experimental Setup 4.2 Evaluation 4.3 Analysis and Discussion 5 Conclusions References Within- and Between-Class Sample Interpolation Based Supervised Metric Learning for Speaker Verification 1 Introduction 2 Method 2.1 Within-Class and Between-Class Points Interpolation Generation (WBIG) 2.2 WBIG for Supervised Contrastive Loss 3 Experiments 3.1 Experimental Settings 3.2 Main Results 3.3 Ablation Experiments 4 Conclusion References Joint Speech and Noise Estimation Using SNR-Adaptive Target Learning for Deep-Learning-Based Speech Enhancement 1 Introduction 2 System Description 2.1 Backgrounds 2.2 SNR-Adaptive Target Learning Strategy 2.3 Joint Speech and Noise Estimation Using SNR-Adaptive Target Learning 3 Experiments and Results Analysis 3.1 Data Corpus 3.2 Implementation Details 3.3 Experiments 4 Conclusion References Data Augmentation by Finite Element Analysis for Enhanced Machine Anomalous Sound Detection 1 Introduction 2 Methods 2.1 Geometric Model 2.2 Model Analysis 2.3 Harmonic Analysis 2.4 Harmonic Acoustics 2.5 Backbone Model 3 Experiments and Results 3.1 Acoustic Simulation 3.2 Dataset 3.3 Result 4 Conclusion References A Fast Sampling Method in Diffusion-Based Dance Generation Models 1 Introduction 2 Related Works 2.1 Denoising Diffusion Probabilistic Models 2.2 Dance Generation Diffusion Model 2.3 Resampling Techniques 3 Proposed Methods 4 Experiments and Results 4.1 Implementation Details 4.2 Quality and Speed 5 Conclusions References End-to-End Streaming Customizable Keyword Spotting Based on Text-Adaptive Neural Search 1 Introduction 2 End-to-End Streaming Framework 2.1 Training Sample Construction 2.2 Keyword-Constrained Attention Based Network 2.3 Cascading Trigger Module 3 Experiment Configuration 3.1 Dataset 3.2 Baseline Models Setup 3.3 Proposed Model Setup 4 Results and Analysis 4.1 Relative Importance of the Modules 4.2 Impact of Negative Samples 4.3 Impact of Multi-label Mechanism 4.4 Comparison of the Model Efficiency 5 Conclusions References The Production of Successive Addition Boundary Tone in Mandarin Preschoolers 1 Introduction 2 Methods 2.1 Participants 2.2 Materials and Procedure 3 Results 3.1 Pitch Development Patterns 3.2 Pitch Range and Register 3.3 Duration 4 Discussion 5 Conclusion References Emotional Support Dialog System Through Recursive Interactions Among Large Language Models 1 Introduction 2 Related Work 2.1 Supportive Psychotherapy 2.2 Emotional Support Dialog Systems for Mental Health 3 System Framework 3.1 Overview 3.2 Dialog Strategy 3.3 Stage 1 - Guided Response and Strategy Generation 3.4 Stage 2 - Cross-Revision and Task Exchange 4 Experimental Setup 4.1 Datasets 4.2 Domain-Specific LLM 4.3 Foundational LLM 5 Result and Discussion 5.1 Automatic Evaluation 5.2 Manual Evaluation 5.3 Case Study 6 Conclusions References Task-Adaptive Generative Adversarial Network Based Speech Dereverberation for Robust Speech Recognition 1 Introduction 2 Conventional GAN 3 Task-Adaptive GAN 3.1 Network Structure 3.2 Loss Function 3.3 Training Process 4 Datasets and Experimental Setup 4.1 Datasets 4.2 Experimental Setup 5 Results and Discussions 6 Conclusions and Future Work References Real-Time Automotive Engine Sound Simulation with Deep Neural Network 1 Introduction 2 Sample-Based Method with Griffin-Lim Algorithm 2.1 Griffin-Lim Algorithm 2.2 Griffin-Lim Algorithm in Frame-Level 3 Procedural Method with Deep Neural Network 4 Experiments and Results 4.1 Dataset and Pre-processing 4.2 Sample-Based Method 4.3 Procedural Method 4.4 Hybrid Process of Synthesis 4.5 Time Cost 5 Conclusions References A Framework Combining Separate and Joint Training for Neural Vocoder-Based Monaural Speech Enhancement 1 Introduction 2 Framework Structure 2.1 Denoising Model 2.2 Model Architecture 3 Experiments 3.1 Data Preparation 3.2 Model Training 3.3 Baseline Model 4 Experimental Results and Analysis 4.1 Results on CSMSC 4.2 Results on Voice Bank+DEMAND 5 Conclusions References Accent-VITS: Accent Transfer for End-to-End TTS 1 Introduction 2 Method 2.1 Pronunciation Encoder 2.2 BN Constraint Module 2.3 Prior Encoder 2.4 Posterior Encoder 2.5 Decoder 2.6 Duration Predictor 2.7 Final Loss 3 Experiments 3.1 Datasets 3.2 Model Configuration 3.3 Subjective Evaluation 3.4 Objective Evaluation 3.5 Ablation Study 4 Conclusions References Multi-branch Network with Cross-Domain Feature Fusion for Anomalous Sound Detection 1 Introduction 2 Proposed Method 2.1 Multi-branch Network Architecture 2.2 Cross-Domain Feature Fusion 2.3 Attentive Sandglass Block 2.4 ArcFace Loss-Based Classifier 3 Experiments and Results 3.1 Dataset and Evaluation Metrics 3.2 Experimental Setting 3.3 Comparison with the State-of-Art Methods 3.4 Ablation Study 4 Conclusion References A Packet Loss Concealment Method Based on the Demucs Network Structure 1 Introduction 2 Modeling of Packet Loss 2.1 Signal Model 2.2 Packet Loss Simulator 3 Proposed Method 3.1 U-Net Architecture 3.2 Loss Function 4 Experimental Results 4.1 Experimental Settings 4.2 Evaluation and Analysis 5 Conclusions References Improving Speech Perceptual Quality and Intelligibility Through Sub-band Temporal Envelope Characteristics 1 Introduction 2 Method 2.1 Temporal Envelope Representation 2.2 The Temporal Envelope Loss Based on Sub-band Weighted 2.3 Joint Loss Function 3 Experiments 3.1 Dataset and Evaluation Metrics 3.2 Experimental Results 4 Conclusion References Adaptive Deep Graph Convolutional Network for Dialogical Speech Emotion Recognition 1 Introduction 2 Methodology 2.1 The Construction of the Dialogue Graph 2.2 Adaptive Deep Graph Convolutional Network 3 Experiments and Analysis 3.1 Experimental Setup 3.2 Experiments Results and Analysis 4 Conclusion References Iterative Noisy-Target Approach: Speech Enhancement Without Clean Speech 1 Introduction 2 Related Work 2.1 Noise2Clean Training 2.2 Noise2Noise Training 2.3 Noisy-Target Training 3 Methodology 3.1 Basic Iterative Method 3.2 Iteration in Step 4 Experiment Setups 4.1 Datasets 4.2 Network Architecture 4.3 Training and Evaluation Details 5 Results 6 Conclusion References Joint Training or Not: An Exploration of Pre-trained Speech Models in Audio-Visual Speaker Diarization 1 Introduction 2 Method 2.1 Overview 2.2 Lip Encoder 2.3 Supervised Pre-trained Models for Audio/Speaker Encoder 2.4 Self-supervised Pre-trained Models for Audio/Speaker Encoder 2.5 Audio-Visual Speaker Decoder 3 Experimental Setup 3.1 Dataset 3.2 Configurations 3.3 Comparison Methods 3.4 Evaluation Metric 4 Results and Analysis 5 Conclusion References Zero-Shot Singing Voice Conversion Based on Timbre Space Modeling and Excitation Signal Control 1 Introduction 2 Proposed Method 2.1 Overall Framework 2.2 Glow-Based Timbre Space Modeling 2.3 Decoder Incorporated with Excitation Signal 2.4 Dual-Decoder for High-Fidelity 48 kHz Waveform Modeling 2.5 Key Shift Based Pitch Mapping Strategy for Conversion Stage 3 Experiments 3.1 Settings 3.2 Results and Analysis 4 Conclusion References A Comparative Study of Pre-trained Audio and Speech Models for Heart Sound Detection 1 Introduction 2 Preliminaries 2.1 Heart Sound 2.2 PANNs 2.3 SSAST 2.4 BEATs 2.5 HuBERT 2.6 WavLM 3 Datasets 3.1 Physionet/CinC 2016 Database 3.2 CirCor DigiScope Dataset 4 Methodology and Experimental Settings 4.1 Comparison of Heart Sound Detection Performance Using Different Pre-trained Models 4.2 Noise Resistance Performance 4.3 Robustness of Models in Real Scenarios 5 Results and Discussions 5.1 Performance of Pre-trained Models with High-Quality Heart Sounds 5.2 Analysis of Model's Noise Resistance Performance 5.3 Performance of Pre-trained Models in Real Scenarios 6 Conclusion References CAM-GUI: A Conversational Assistant on Mobile GUI 1 Introduction 2 System Architecture 2.1 Interaction Module 2.2 Decision Module 3 Dataset 3.1 Task Design 3.2 Collection Platform 3.3 Data Collection 3.4 Quality Check and Data Cleaning 4 Experiment Demonstrations 5 User Study 6 Related Work 7 Conclusion and Future Work A Ethics and Broader Impact Statement B Task Templates C An Example for Dialog Traces D User Study E Demonstration Video References A Pilot Study on the Prosodic Factors Influencing Voice Attractiveness of AI Speech 1 Introduction 2 AI Voice Attractiveness Comparative Evaluation 2.1 Attractiveness of Four AI Voices 2.2 Comparison of AI Voices Power Spectra 3 How Acoustic Prosodic Parameters Affect AI Voice Attractiveness 3.1 Fundamental Frequency 3.2 Intonation Variability 3.3 Duration 4 The Impact of Finnish Language Prosodic Feature Variations on AI Voice Attractiveness 4.1 Fundamental Frequency 4.2 Duration 5 Conclusion References The DKU-MSXF Diarization System for the VoxCeleb Speaker Recognition Challenge 2023 1 Introduction 2 Dataset Description 3 Model Configuration 3.1 VAD and OSD 3.2 Speaker Embedding 3.3 Clustering-Based Diarization 3.4 TSVAD-Based Diarization 4 Experimental Results 5 Conclusions References Chinese EFL Learners’ Auditory and Visual Perception of English Statement and Question Intonations: The Effect of Lexical Stress 1 Introduction 2 Methods 2.1 Participant 2.2 Stimuli 2.3 Procedure 2.4 Data Analysis 3 Results 3.1 Effects of Stress Position 3.2 Effects of English Proficiency 3.3 Effects of Visual Cues 4 Discussion 5 Conclusion References An Improved System for Partially Fake Audio Detection Using Pre-trained Model 1 Introduction 2 Proposed System Structure 2.1 Pre-trained Model 2.2 Feature Extractor 2.3 Data Augmentation 2.4 Networks 2.5 Temporal Contrastival Loss 2.6 Post-processing 3 Experiments 3.1 Dataset 3.2 Measurement 3.3 Experiments and Results 4 Conclusion References Leveraging Synthetic Speech for CIF-Based Customized Keyword Spotting 1 Introduction 2 Related Work 2.1 Text-to-Speech 2.2 Continuous Integrate-and-Fire 3 Method 3.1 TTS Generator 3.2 Shared Encoder 3.3 Pattern Extractor 3.4 Verifier 3.5 Training Process 4 Experiment 4.1 Dataset 4.2 Model Details 4.3 Evaluation 4.4 Results 5 Conclusions References Author Index