Random Signal Processing

"This book covers random signals and random processes along with estimation of probability density function, estimation of energy spectral density and power spectral density. The properties of random processes and signal modelling are discussed with basic communication theory estimation and detection. MATLAB simulations are included for each concept with output of the program with case studies and project ideas. The chapters progressively introduce and explain the concepts of random signals and cover multiple applications for signal processing. The book is designed to cater to a wide audience starting from the undergraduates (electronics, electrical, instrumentation, computer, and telecommunication engineering) to the researchers working in the pertinent fields. Key Features: " Aimed at random signal processing with parametric signal processing-using appropriate segment size." Covers speech, image, medical images, EEG and ECG signal processing." Reviews optimal detection and estimation." Discusses parametric modeling and signal processing in transform domain." Includes MATLAB codes and relevant exercises, case studies and solved examples including multiple choice questions"--Provided by publisher Cover 1 Half Title 2 Title Page 4 Copyright Page 5 Dedication 6 Contents 8 Preface 16 Acknowledgments 20 About the Author 21 1. Introduction to Random Signals 24 Learning Objectives 24 1.1 Introduction to Set Theory 24 1.1.1 Union and Intersection 26 1.1.2 Algebra of Sets 27 1.1.3 De Morgan’s Laws 27 1.1.4 Duality Principle 28 Concept Check 28 1.2 Probability 28 1.2.1 Conditional Probability 30 1.2.2 Bayes’ Theorem 35 1.2.2.1 Alternative Statement of Bayes’ Theorem 37 1.2.3 Mutually Exclusive and Independent Events 39 Concept Check 40 1.3 Random Variable 41 1.3.1 Cumulative Distribution Function (CDF) 41 1.3.2 Probability Density Function (pdf) 44 Concept Check 48 1.4 Standard Distribution Functions 49 1.4.1 Probability Distribution Functions for Continuous Variables 49 1.4.2 Probability Distribution Functions for Discrete Variables 56 1.4.2.1 Permutations 56 1.4.2.2 Combinations 57 1.4.2.3 Bernoulli’s Trials 57 1.4.2.4 Binomial Distribution 57 1.4.2.5 Poisson Distribution 59 Concept Check 62 1.5 Central Limit Theorem, Chi-Square Test, and Kolmogorov–Smirnov Test 63 1.5.1 Central Limit Theorem 63 1.5.2 Computer Generation of a Gaussian Distributed Random Variable 63 1.5.3 Chi-Square Test 65 1.5.4 Kolmogorov–Smirnov Test 66 Concept Check 67 Summary 68 Key Terms 69 Multiple-Choice Questions 69 Review Questions 71 Problems 71 Answers 73 Multiple-Choice Questions 73 Problems 73 2. Properties of Random Variables 76 Learning Objectives 76 2.1 Statistical Properties of Random Variables 76 Concept Check 86 2.2 Functions for Finding Moments 86 Concept Check 89 2.3 Transformations of a Random Variable 89 2.3.1 Monotonic Transformations of a Random Variable 89 2.3.2 Multiple-Valued Transformations of a Random Variable 91 2.3.3 Computer Generation of a Transformed Variable 94 Concept Check 96 2.4 Computation of the Energy Density Spectrum of a Deterministic Signal 96 2.4.1 Estimation of Power Density Spectrum of Random Signal 104 2.4.2 Use of Discrete Fourier Transform (DFT) for Power Spectrum Estimation 106 Concept Check 107 Summary 107 Key Terms 108 Multiple-Choice Questions 109 Review Questions 111 Problems 111 Answers 113 Multiple-Choice Questions 113 Problems 114 3. Multiple Random Variables and Random Process 118 Learning Objectives 118 3.1 Multiple Random Variables 118 3.1.1 Marginal Density Functions and Statistical Independence 120 3.1.2 Operations on Multiple Random Variables 121 Concept Check 136 3.2 Modeling a Random Signal 137 3.2.1 AR, MA, and ARMA Modeling 137 Concept Check 138 3.3 Random Processes 138 3.3.1 Stationary and Nonstationary Process 139 3.3.2 Stationary Processes 139 3.3.3 N[sup(th)] Order or Strict Sense Stationary Process 140 Concept Check 141 Summary 141 Key Terms 142 Multiple-Choice Questions 142 Review Questions 145 Problems 145 Answers 147 Multiple-Choice Questions 147 Problems 148 4. Detection and Estimation 150 4.1 Basics of Communication Theory 150 4.1.1 Transmitter 150 4.1.1.1 Justicfiation for Sampling and Quantization of Analog Signals 150 4.1.2 Channel 151 4.1.3 Receiver 151 Concept Check 152 4.2 Linear but Time Varying Systems 152 4.2.1 Filter Characteristics of Linear Systems 152 4.2.2 Distortionless Transmission through a System 154 4.2.2.1 Signal Bandwidth 155 4.2.3 Ideal Low-Pass Filter, High-Pass Filter, and Bandpass Filter Characteristics 155 4.2.4 Causality and Paley–Wiener Criteria 158 4.2.4.1 Statement of the Theorem 159 4.2.5 Relation between Bandwidth and Rise Time 160 Concept Check 160 4.3 Optimum Detection 161 4.3.1 Weighted Probabilities and Hypothesis Testing 161 4.3.2 Bayes Criterion 163 4.3.3 Minimax Criterion 164 4.3.4 Neyman–Pearson Criterion 165 4.3.5 Receiver Operating Characteristics 165 Concept Check 166 4.4 Estimation Theory 166 Concept Check 169 Summary 169 Keywords 170 Multiple-Choice Questions 171 Review Questions 174 Answers 175 Multiple-Choice Questions 175 5. Fundamentals of Speech Processing 176 Learning Objectives 176 5.1 LTI and LTV Models for Speech Production 177 5.1.1 LTI Model for Speech 177 5.1.2 Nature of Speech Signal 178 5.1.3 LTV Model 179 Concept Check 179 5.2 Voiced and Unvoiced Decision-Making 179 Analysis of the MATLAB Program Output 183 Concept Check 188 5.3 Audio File Formats—Nature of .wav File 189 Concept Check 191 5.4 Extraction of Fundamental Frequency 191 5.4.1 Fundamental Frequency or Pitch Frequency 191 5.4.1.1 Autocorrelation Method for Finding Pitch Period of a Voiced Speech Segment 191 5.4.1.2 AMDF Method for Finding Pitch Period of a Voiced Speech Segment 195 5.4.2 Pitch Contour 197 5.4.3 Pitch Period Measurement in Spectral Domain 197 5.4.3.1 Spectral Autocorrelation Method for Pitch Measurement 199 5.4.4 Cepstral Domain 202 5.4.5 Pitch Period Measurement Using Cepstral Domain 204 5.4.5.1 Interpretation of the Result 208 5.4.5.2 Interpretation of the Result 212 Concept Check 213 5.5 Formants and Relation of Formants with LPC 214 5.5.1 Practical Considerations 214 Concept Check 215 5.6 Evaluation of Formants 215 5.6.1 Evaluation of Formants Using Cepstrum 215 5.6.1.1 Evaluation of Formants for Voiced Speech Segment 215 5.6.1.2 Evaluation of Formants for Unvoiced Segment 220 5.6.2 Evaluation of Formants Using the Log Spectrum 223 5.6.2.1 Evaluation of Formants for Voiced Speech Segment 224 5.6.2.2 Evaluation of Formants for Unvoiced Segment 226 Concept Check 228 5.7 Evaluation of MFCC 229 5.7.1 Homomorphic Processing 229 5.7.2 The Auditory System as a Filter Bank 232 5.7.3 Mel Frequency Cepstral Coefcfiients 236 Concept Check 241 5.8 Evaluation of LPC 242 5.8.1 Forward Linear Prediction 242 5.8.2 Autocorrelation Method 245 Concept Check 251 Summary 251 Key Terms 252 Multiple-Choice Questions 253 Review Questions 255 Problems 256 Answers 257 Multiple-Choice Questions 257 Problems 258 6. Spectral Estimation of Random Signals 260 Learning Objectives 260 6.1 Estimation of Density Spectrum 261 6.1.1 Classicfiation of Signals 261 6.1.2 Power Spectral Density and Energy Spectral Density 262 6.1.2.1 Computation of Energy Density Spectrum of Deterministic Signals 262 6.1.2.2 Estimation of Power Density Spectrum of Random Signals 264 Concept Check 265 6.2 Nonparametric Methods 265 6.2.1 Use of DFT for Power Spectrum Estimation 266 6.2.2 Bartlett Method 268 6.2.3 Welch Method 270 6.2.4 Blackman–Tukey Method 272 6.2.5 Performance Comparison of Nonparametric Methods 273 Concept Check 275 6.3 Parametric Methods 275 6.3.1 Power Spectrum Estimation Using AR Model Parameters 276 6.3.2 Burg’s Method for Power Spectrum Estimation (Maximum Entropy Method—MEM) 277 6.3.3 Power Spectrum Estimation Using ARMA Model 280 6.3.4 Power Spectrum Estimation Using MA Model 280 Concept Check 280 6.4 Other Spectral Estimation Methods 280 6.4.1 Minimum Variance Power Spectrum Estimation 281 6.4.2 Eigen Analysis Algorithm for Spectrum Estimation 281 Concept Check 285 6.5 Evaluation of Formants Using the Power Spectral Density Estimate 285 6.5.1 Interpretation of the Results 286 Concept Check 288 6.6 Evaluation of Cepstrum 290 6.7 Evaluation of Higher Order Spectra 291 6.7.1 Cumulant Spectra 292 6.7.1.1 Indirect Method 293 6.7.1.2 Direct Method 294 Concept Check 299 Summary 300 Key Terms 301 Multiple-Choice Questions 301 Review Questions 304 Problems 305 Suggested Projects 305 Answers 306 Multiple Choice Questions 306 Problems 307 7. Statistical Speech Processing 308 Learning Objectives 308 7.1 Measurement of Statistical Parameters of Speech 308 Concept Check 311 7.2 Dynamic Time Warping 311 7.2.1 Linear Time Warping 312 7.2.2 Dynamic Time Warping 312 7.3 Statistical Sequence Recognition for Automatic Speech Recognition (ASR) 316 7.3.1 Bayes Rule 316 7.3.2 Hidden Markov Model 317 Concept Check 319 7.4 Statistical Pattern Recognition and Parameter Estimation 319 7.4.1 Statistical Parameter Estimation 320 7.4.2 Acoustic Probability Estimation for ASR 321 Concept Check 321 7.5 VQ-HMM-Based Speech Recognition 321 7.5.1 HMM Specicfiation and Model Training 323 7.5.1.1 Forward Algorithm 324 7.5.1.2 Backward Algorithm 325 7.5.1.3 Viterbi Algorithm 325 7.5.1.4 Baum–Welch Algorithm 326 7.5.1.5 Posterior Decoding 327 Concept Check 327 7.6 Discriminant Acoustic Probability Estimation 328 7.6.1 Discriminant Training 328 7.6.2 Use of Neural Networks 329 Concept Check 330 Summary 330 Key Terms 331 Multiple-Choice Questions 331 Review Questions 333 Problems 333 Answers 334 Multiple-Choice Questions 334 8. Transform Domain Speech Processing 336 Learning Objectives 336 8.1 Short Segment Analysis of Speech 337 8.1.1 Pitch Synchronous Analysis of Speech 337 Concept Check 339 8.2 Use of Transforms 339 8.2.1 Discrete Cosine Transform 340 Concept Check 343 8.3 Applications of DCT for Speech Processing 343 8.3.1 Signal Coding 343 8.3.2 Signal Filtering 344 8.3.3 Sampling Rate Conversion and Resizing 346 8.3.4 Feature Extraction and Recognition 348 Concept Check 349 8.4 Short-Time Fourier Transform (STFT) 349 Concept Check 351 8.5 Wavelet Transform 352 Concept Check 354 8.6 Haar Wavelet and Multiresolution Analysis 354 8.6.1 Multiresolution Analysis 357 Concept Check 361 8.7 Daubechies Wavelets 362 8.7.1 Matrix Multiplication Method for Computation of WT 363 8.7.2 Number of Operations 365 8.7.3 Time Band Width Product 365 Concept Check 366 8.8 Some Other Standard Wavelets 366 8.8.1 Mexican Hat Function 366 8.8.2 A Modulated Gaussian 366 8.8.3 Spline and Battle–Lemarie Wavelets 366 8.8.4 Biorthogonal Wavelets 367 8.8.5 Cohen–Daubechies–Feauveau Family of Biorthogonal Spline Wavelets 367 8.8.6 Wavelet Packets 368 8.8.6 1 Full Wavelet Packet Decomposition 368 Concept Check 369 8.9 Applications of WT 369 8.9.1 Denoising Using DWT 369 8.9.2 Signal Compression 372 8.9.3 Signal Filtering 375 8.9.4 Sampling Rate Conversion 376 Concept Check 378 Summary 379 Key Terms 380 Multiple-Choice Questions 381 Review Questions 383 Problems 384 Suggested Projects 385 Answers 385 Multiple-Choice Questions 385 Problems 386 9. Image Processing Techniques 388 Learning Objectives 388 9.1 Image Representation and Spatial Filtering 388 9.1.1 Edge Detection Using Spatial Filtering 392 9.1.2 Laplacian Mask 397 9.1.3 Laplacian of Gaussian 399 Concept Check 402 9.2 Transformations on Image 403 9.2.1 Linear Transformations 403 9.2.2 Gray-Level Slicing 407 9.2.3 Bit-Plane Slicing 409 9.2.3 Nonlinear Transformations 410 9.3 Histogram Equalization 412 9.3.1 Histogram Evaluation 412 9.3.2 Mapping the pdf Value with CDF 414 9.3.3 Histogram Equalization 415 9.3.4 Statistical Image Processing 418 Concept Check 418 9.4 Transform Domain Image Processing 419 9.4.1 Image Processing Using DCT 419 9.4.2 Image Processing Using WT 424 Concept Check 430 Summary 431 Key Terms 431 Multiple-Choice Questions 432 Review Questions 434 Problems 435 Answers 435 Multiple-Choice Questions 435 Problems 436 10. Applications of Random Signal Processing 438 Learning Objectives 438 10.1 Case Study 1: Handwritten Character Recognition 438 10.1.1 Components of an OCR System 438 10.1.2 Challenges in Devanagari Handwriting Recognition 439 10.1.3 Tree Classicfiation Based on Structural Features 441 10.1.4 Recognition Using Neural Network 444 10.2 Case Study 2: Writer Identicfiation and Vericfiation 446 10.2.1 Introduction 446 10.2.2 Importance of Writer Identicfiation and Vericfiation 447 10.2.3 Main Factors Discriminating Handwritings 447 10.2.4 Factors Affecting Handwriting 447 10.2.4.1 System for Writer Identicfiation and Vericfiation 448 10.2.4.2 Handwriting Acquisition 448 10.2.4.3 Preprocessing 448 10.2.4.4 Feature Extraction 448 10.2.5 Off-Line English Handwriting Databases 451 10.2.5.1 Writer Vericfiation System 453 Index 456