Artificial Neural Networks with Java : Tools for Building Neural Network Applications

Develop neural network applications using the Java environment. After learning the rules involved in neural network processing, this second edition shows you how to manually process your first neural network example. The book covers the internals of front and back propagation and helps you understand the main principles of neural network processing. You also will learn how to prepare the data to be used in neural network development and you will be able to suggest various techniques of data preparation for many unconventional tasks. This book discusses the practical aspects of using Java for neural network processing. You will know how to use the Encog Java framework for processing large-scale neural network applications. Also covered is the use of neural networks for approximation of non-continuous functions. In addition to using neural networks for regression, this second edition shows you how to use neural networks for computer vision. It focuses on image recognition such as the classification of handwritten digits, input data preparation and conversion, and building the conversion program. And you will learn about topics related to the classification of handwritten digits such as network architecture, program code, programming logic, and execution. The step-by-step approach taken in the book includes plenty of examples, diagrams, and screenshots to help you grasp the concepts quickly and easily. What You Will Learn Use Java for the development of neural network applications Prepare data for many different tasks Carry out some unusual neural network processing Use a neural network to process non-continuous functions Develop a program that recognizes handwritten digits Who This Book Is For Intermediate machine learning and deep learning developers who are interested in switching to Java Table of Contents 5 About the Author 11 About the Technical Reviewers 12 Acknowledgments 13 Introduction 14 Part I: Getting Started with Neural Networks 16 Chapter 1: Learning About Neural Networks 17 Biological and Artificial Neurons 18 Activation Functions 19 Summary 21 Chapter 2: Internal Mechanics of Neural Network Processing 22 Function to Be Approximated 22 Network Architecture 23 Forward Pass Calculation 25 Input Record 1 26 Input Record 2 27 Input Record 3 27 Input Record 4 28 Back-Propagation Pass 29 Function Derivative and Function Divergent 30 Most Commonly Used Function Derivatives 31 Summary 32 Chapter 3: Manual Neural Network Processing 33 Example: Manual Approximation of a Function at a Single Point 33 Building the Neural Network 34 Forward Pass Calculation 36 Hidden Layers 37 Output Layer 37 Backward Pass Calculation 38 Calculating Weight Adjustments for the Output-Layer Neurons 38 Calculating Adjustment for W211 39 Calculating Adjustment for W212 40 Calculating Adjustment for W213 41 Calculating Weight Adjustments for Hidden-Layer Neurons 42 Calculating Adjustment for W111 42 Calculating Adjustment for W112 43 Calculating Adjustment for W121 44 Calculating Adjustment for W122 44 Calculating Adjustment for W131 45 Calculating Adjustment for W132 46 Updating Network Biases 47 Back to the Forward Pass 48 Hidden Layers 48 Output Layer 49 Matrix Form of Network Calculation 51 Digging Deeper 52 Mini-Batches and Stochastic Gradient 54 Summary 55 Part II: Neural Network Java Development Environment 56 Chapter 4: Configuring Your Development Environment 57 Installing the Java Environment and NetBeans on Your Windows Machine 57 Installing the Encog Java Framework 61 Installing the XChart Package 62 Summary 63 Chapter 5: Neural Networks Development Using the Java Encog Framework 64 Example: Function Approximation Using Java Environment 64 Network Architecture 66 Normalizing the Input Datasets 67 Building the Java Program That Normalizes Both Datasets 67 Building the Neural Network Processing Program 78 Program Code 88 Debugging and Executing the Program 110 Processing Results for the Training Method 112 Testing the Network 113 Testing Results 117 Digging Deeper 118 Summary 119 Chapter 6: Neural Network Prediction Outside of the Training Range 120 Example: Approximating Periodic Functions Outside of the Training Range 120 Network Architecture for the Example 125 Program Code for the Example 125 Testing the Network 143 Example: Correct Way of Approximating Periodic Functions Outside of the Training Range 145 Preparing the Training Data 145 Network Architecture for the Example 149 Program Code for Example 150 Training Results for Example 172 Log of Testing Results for Example 3 173 Summary 175 Chapter 7: Processing Complex Periodic Functions 176 Example: Approximation of a Complex Periodic Function 176 Data Preparation 179 Reflecting Function Topology in the Data 181 Network Architecture 187 Program Code 188 Training the Network 212 Testing the Network 214 Digging Deeper 217 Summary 218 Chapter 8: Approximating Noncontinuous Functions 219 Example: Approximating Noncontinuous Functions 219 Network Architecture 223 Program Code 224 Code Fragments for the Training Process 238 Unsatisfactory Training Results 242 Approximating the Noncontinuous Function Using the Micro-Batch Method 245 Program Code for Micro-Batch Processing 247 Program Code for the getChart() Method 270 Code Fragment 1 of the Training Method 276 Code Fragment 2 of the Training Method 277 Training Results for the Micro-Batch Method 283 Testing the Processing Logic 289 Testing the Results for the Micro-Batch Method 293 Digging Deeper 295 Summary 302 Chapter 9: Approximation of Continuous Functions with Complex Topology 303 Example: Approximation of Continuous Functions with Complex Topology Using a Conventional Neural Network Process 303 Network Architecture for the Example 306 Program Code for the Example 307 Training Processing Results for the Example 321 Approximation of Continuous Functions with Complex Topology Using the Micro-Batch Method 325 Program Code for the Example Using the Micro-Batch Method 329 Example: Approximation of Spiral-like Functions 356 Network Architecture for the Example 360 Program Code for Example 361 Approximation of the Same Functions Using Micro-Batch Method 378 Summary 408 Chapter 10: Using Neural Networks for the Classification of Objects 409 Example: Classification of Records 409 Training Dataset 411 Network Architecture 415 Testing Dataset 415 Program Code for Data Normalization 416 Program Code for Classification 422 Training Results 452 Testing Results 462 Summary 463 Chapter 11: The Importance of Selecting the Correct Model 464 Example: Predicting Next Month’s Stock Market Price 464 Including the Function Topology in the Dataset 478 Building Micro-Batch Files 486 Network Architecture 491 Program Code 492 Training Process 526 Training Results 528 Testing Dataset 532 Testing Logic 543 Testing Results 553 Analyzing Testing Results 560 Summary 562 Chapter 12: Approximation Functions in 3D Space 563 Example: Approximation Functions in 3D Space 564 Data Preparation 564 Network Architecture 569 Program Code 570 Processing Results 586 Summary 593 Part III: Introduction to Computer Vision 594 Chapter 13: Image Recognition 595 Classification of Handwritten Digits 596 Preparing the Input Data 597 Input Data Conversion 598 Building the Conversion Program 599 Summary 610 Chapter 14: Classification of Handwritten Digits 611 Network Architecture 611 Program Code 613 Programming Logic 626 Execution 628 Convolution Neural Network 629 Summary 630 Index 631