Data Mining Algorithms : Explained Using R

**__Data Mining Algorithms__** is a practical, technically-oriented guide to data mining algorithms that covers the most important algorithms for building classification, regression, and clustering models, as well as techniques used for attribute selection and transformation, model quality evaluation, and creating model ensembles. The author presents many of the important topics and methodologies widely used in data mining, whilst demonstrating the internal operation and usage of data mining algorithms using examples in R. This Book Narrows Down The Scope Of Data Mining By Adopting A Heavily Modeling-oriented Perspective-- Machine Generated Contents Note: Part I Preliminaries -- 1. Tasks -- 1.1. Introduction -- 1.1.1. Knowledge -- 1.1.2. Inference -- 1.2. Inductive Learning Tasks -- 1.2.1. Domain -- 1.2.2. Instances -- 1.2.3. Attributes -- 1.2.4. Target Attribute -- 1.2.5. Input Attributes -- 1.2.6. Training Set -- 1.2.7. Model -- 1.2.8. Performance -- 1.2.9. Generalization -- 1.2.10. Overfitting -- 1.2.11. Algorithms -- 1.2.12. Inductive Learning As Search -- 1.3. Classification -- 1.3.1. Concept -- 1.3.2. Training Set -- 1.3.3. Model -- 1.3.4. Performance -- 1.3.5. Generalization -- 1.3.6. Overfitting -- 1.3.7. Algorithms -- 1.4. Regression -- 1.4.1. Target Function -- 1.4.2. Training Set -- 1.4.3. Model -- 1.4.4. Performance -- 1.4.5. Generalization -- 1.4.6. Overfitting -- 1.4.7. Algorithms -- 1.5. Clustering -- 1.5.1. Motivation -- 1.5.2. Training Set -- 1.5.3. Model -- 1.5.4. Crisp Vs. Soft Clustering -- 1.5.5. Hierarchical Clustering -- 1.5.6. Performance -- 1.5.7. Generalization -- 1.5.8. Algorithms. 1.5.9. Descriptive Vs. Predictive Clustering -- 1.6. Practical Issues -- 1.6.1. Incomplete Data -- 1.6.2. Noisy Data -- 1.7. Conclusion -- 1.8. Further Readings -- References -- 2. Basic Statistics -- 2.1. Introduction -- 2.2. Notational Conventions -- 2.3. Basic Statistics As Modeling -- 2.4. Distribution Description -- 2.4.1. Continuous Attributes -- 2.4.2. Discrete Attributes -- 2.4.3. Confidence Intervals -- 2.4.4.m-estimation -- 2.5. Relationship Detection -- 2.5.1. Significance Tests -- 2.5.2. Continuous Attributes -- 2.5.3. Discrete Attributes -- 2.5.4. Mixed Attributes -- 2.5.5. Relationship Detection Caveats -- 2.6. Visualization -- 2.6.1. Boxplot -- 2.6.2. Histogram -- 2.6.3. Barplot -- 2.7. Conclusion -- 2.8. Further Readings -- References -- Part Ii Classification -- 3. Decision Trees -- 3.1. Introduction -- 3.2. Decision Tree Model -- 3.2.1. Nodes And Branches -- 3.2.2. Leaves -- 3.2.3. Split Types -- 3.3. Growing -- 3.3.1. Algorithm Outline. 3.3.2. Class Distribution Calculation -- 3.3.3. Class Label Assignment -- 3.3.4. Stop Criteria -- 3.3.5. Split Selection -- 3.3.6. Split Application -- 3.3.7.complete Process -- 3.4. Pruning -- 3.4.1. Pruning Operators -- 3.4.2. Pruning Criterion -- 3.4.3. Pruning Control Strategy -- 3.4.4. Conversion To Rule Sets -- 3.5. Prediction -- 3.5.1. Class Label Prediction -- 3.5.2. Class Probability Prediction -- 3.6. Weighted Instances -- 3.7. Missing Value Handling -- 3.7.1. Fractional Instances -- 3.7.2. Surrogate Splits -- 3.8. Conclusion -- 3.9. Further Readings -- References -- 4. Naive Bayes Classifier -- 4.1. Introduction -- 4.2. Bayes Rule -- 4.3. Classification By Bayesian Inference -- 4.3.1. Conditional Class Probability -- 4.3.2. Prior Class Probability -- 4.3.3. Independence Assumption -- 4.3.4. Conditional Attribute Value Probabilities -- 4.3.5. Model Construction -- 4.3.6. Prediction -- 4.4. Practical Issues -- 4.4.1. Zero And Small Probabilities. 4.4.2. Linear Classification -- 4.4.3. Continuous Attributes -- 4.4.4. Missing Attribute Values -- 4.4.5. Reducing Naivety -- 4.5. Conclusion -- 4.6. Further Readings -- References -- 5. Linear Classification -- 5.1. Introduction -- 5.2. Linear Representation -- 5.2.1. Inner Representation Function -- 5.2.2. Outer Representation Function -- 5.2.3. Threshold Representation -- 5.2.4. Logit Representation -- 5.3. Parameter Estimation -- 5.3.1. Delta Rule -- 5.3.2. Gradient Descent -- 5.3.3. Distance To Decision Boundary -- 5.3.4. Least Squares -- 5.4. Discrete Attributes -- 5.5. Conclusion -- 5.6. Further Readings -- References -- 6. Misclassification Costs -- 6.1. Introduction -- 6.2. Cost Representation -- 6.2.1. Cost Matrix -- 6.2.2. Per-class Cost Vector -- 6.2.3. Instance-specific Costs -- 6.3. Incorporating Misclassification Costs -- 6.3.1. Instance Weighting -- 6.3.2. Instance Resampling -- 6.3.3. Minimum-cost Rule -- 6.3.4. Instance Relabeling. 6.4. Effects Of Cost Incorporation -- 6.5. Experimental Procedure -- 6.6. Conclusion -- 6.7. Further Readings -- References -- 7. Classification Model Evaluation -- 7.1. Introduction -- 7.1.1. Dataset Performance -- 7.1.2. Training Performance -- 7.1.3. True Performance -- 7.2. Performance Measures -- 7.2.1. Misclassification Error -- 7.2.2. Weighted Misclassification Error -- 7.2.3. Mean Misclassification Cost -- 7.2.4. Confusion Matrix -- 7.2.5. Roc Analysis -- 7.2.6. Probabilistic Performance Measures -- 7.3. Evaluation Procedures -- 7.3.1. Model Evaluation Vs. Modeling Procedure Evaluation -- 7.3.2. Evaluation Caveats -- 7.3.3. Hold-out -- 7.3.4. Cross-validation -- 7.3.5. Leave-one-out -- 7.3.6. Bootstrapping -- 7.3.7. Choosing The Right Procedure -- 7.3.8. Evaluation Procedures For Temporal Data -- 7.4. Conclusion -- 7.5. Further Readings -- References -- Part Iii Regression -- 8. Linear Regression -- 8.1. Introduction -- 8.2. Linear Representation. 8.2.1. Parametric Representation -- 8.2.2. Linear Representation Function -- 8.2.3. Nonlinear Representation Functions -- 8.3. Parameter Estimation -- 8.3.1. Mean Square Error Minimization -- 8.3.2. Delta Rule -- 8.3.3. Gradient Descent -- 8.3.4. Least Squares -- 8.4. Discrete Attributes -- 8.5. Advantages Of Linear Models -- 8.6. Beyond Linearity -- 8.6.1. Generalized Linear Representation -- 8.6.2. Enhanced Representation -- 8.6.3. Polynomial Regression -- 8.6.4. Piecewise-linear Regression -- 8.7. Conclusion -- 8.8. Further Readings -- References -- 9. Regression Trees -- 9.1. Introduction -- 9.2. Regression Tree Model -- 9.2.1. Nodes And Branches -- 9.2.2. Leaves -- 9.2.3. Split Types -- 9.2.4. Piecewise-constant Regression -- 9.3. Growing -- 9.3.1. Algorithm Outline -- 9.3.2. Target Function Summary Statistics -- 9.3.3. Target Value Assignment -- 9.3.4. Stop Criteria -- 9.3.5. Split Selection -- 9.3.6. Split Application -- 9.3.7.complete Process -- 9.4. Pruning. 9.4.1. Pruning Operators -- 9.4.2. Pruning Criterion -- 9.4.3. Pruning Control Strategy -- 9.5. Prediction -- 9.6. Weighted Instances -- 9.7. Missing Value Handling -- 9.7.1. Fractional Instances -- 9.7.2. Surrogate Splits -- 9.8. Piecewise Linear Regression -- 9.8.1. Growing -- 9.8.2. Pruning -- 9.8.3. Prediction -- 9.9. Conclusion -- 9.10. Further Readings -- References -- 10. Regression Model Evaluation -- 10.1. Introduction -- 10.1.1. Dataset Performance -- 10.1.2. Training Performance -- 10.1.3. True Performance -- 10.2. Performance Measures -- 10.2.1. Residuals -- 10.2.2. Mean Absolute Error -- 10.2.3. Mean Square Error -- 10.2.4. Root Mean Square Error -- 10.2.5. Relative Absolute Error -- 10.2.6. Coefficient Of Determination -- 10.2.7. Correlation -- 10.2.8. Weighted Performance Measures -- 10.2.9. Loss Functions -- 10.3. Evaluation Procedures -- 10.3.1. Hold-out -- 10.3.2. Cross-validation -- 10.3.3. Leave-one-out -- 10.3.4. Bootstrapping -- 10.3.5. Choosing The Right Procedure. 10.4. Conclusion -- 10.5. Further Readings -- References -- Part Iv Clustering -- 11.(dis)similarity Measures -- 11.1. Introduction -- 11.2. Measuring Dissimilarity And Similarity -- 11.3. Difference-based Dissimilarity -- 11.3.1. Euclidean Distance -- 11.3.2. Minkowski Distance -- 11.3.3. Manhattan Distance -- 11.3.4. Canberra Distance -- 11.3.5. Chebyshev Distance -- 11.3.6. Hamming Distance -- 11.3.7. Gower's Coefficient -- 11.3.8. Attribute Weighting -- 11.3.9. Attribute Transformation -- 11.4. Correlation-based Similarity -- 11.4.1. Discrete Attributes -- 11.4.2. Pearson's Correlation Similarity -- 11.4.3. Spearman's Correlation Similarity -- 11.4.4. Cosine Similarity -- 11.5. Missing Attribute Values -- 11.6. Conclusion -- 11.7. Further Readings -- References -- 12.k-centers Clustering -- 12.1. Introduction -- 12.1.1. Basic Principle -- 12.1.2.(dis)similarity Measures -- 12.2. Algorithm Scheme -- 12.2.1. Initialization -- 12.2.2. Stop Criteria -- 12.2.3. Cluster Formation. 12.2.4. Implicit Cluster Modeling -- 12.2.5. Instantiations -- 12.3.k-means -- 12.3.1. Center Adjustment -- 12.3.2. Minimizing Dissimilarity To Centers -- 12.4. Beyond Means -- 12.4.1.k-medians -- 12.4.2.k-medoids -- 12.5. Beyond (fixed) K -- 12.5.1. Multiple Runs -- 12.5.2. Adaptive K-centers -- 12.6. Explicit Cluster Modeling -- 12.7. Conclusion -- 12.8. Further Readings -- References -- 13. Hierarchical Clustering -- 13.1. Introduction -- 13.1.1. Basic Approaches -- 13.1.2.(dis)similarity Measures -- 13.2. Cluster Hierarchies -- 13.2.1. Motivation -- 13.2.2. Model Representation -- 13.3. Agglomerative Clustering -- 13.3.1. Algorithm Scheme -- 13.3.2. Cluster Linkage -- 13.4. Divisive Clustering -- 13.4.1. Algorithm Scheme -- 13.4.2. Wrapping A Flat Clustering Algorithm -- 13.4.3. Stop Criteria -- 13.5. Hierarchical Clustering Visualization -- 13.6. Hierarchical Clustering Prediction -- 13.6.1. Cutting Cluster Hierarchies -- 13.6.2. Cluster Membership Assignment. 13.7. Conclusion -- 13.8. Further Readings -- References -- 14. Clustering Model Evaluation -- 14.1. Introduction -- 14.1.1. Dataset Performance -- 14.1.2. Training Performance -- 14.1.3. True Performance -- 14.2. Per-cluster Quality Measures -- 14.2.1. Diameter -- 14.2.2. Separation -- 14.2.3. Isolation -- 14.2.4. Silhouette Width -- 14.2.5. Davies -- Bouldin Index -- 14.3. Overall Quality Measures -- 14.3.1. Dunn Index -- 14.3.2. Average Davies -- Bouldin Index -- 14.3.3.c Index -- 14.3.4. Average Silhouette Width -- 14.3.5. Loglikelihood -- 14.4. External Quality Measures -- 14.4.1. Misclassification Error -- 14.4.2. Rand Index -- 14.4.3. General Relationship Detection Measures -- 14.5. Using Quality Measures -- 14.6. Conclusion -- 14.7. Further Readings -- References -- Part V Getting Better Models -- 15. Model Ensembles -- 15.1. Introduction -- 15.2. Model Committees -- 15.3. Base Models -- 15.3.1. Different Training Sets -- 15.3.2. Different Algorithms. 15.3.3. Different Parameter Setups -- 15.3.4. Algorithm Randomization -- 15.3.5. Base Model Diversity -- 15.4. Model Aggregation -- 15.4.1. Voting/averaging -- 15.4.2. Probability Averaging -- 15.4.3. Weighted Voting/averaging -- 15.4.4. Using As Attributes -- 15.5. Specific Ensemble Modeling Algorithms -- 15.5.1. Bagging -- 15.5.2. Stacking -- 15.5.3. Boosting -- 15.5.4. Random Forest -- 15.5.5. Random Naive Bayes -- 15.6. Quality Of Ensemble Predictions -- 15.7. Conclusion -- 15.8. Further Readings -- References -- 16. Kernel Methods -- 16.1. Introduction -- 16.2. Support Vector Machines -- 16.2.1. Classification Margin -- 16.2.2. Maximum-margin Hyperplane -- 16.2.3. Primal Form -- 16.2.4. Dual Form -- 16.2.5. Soft Margin -- 16.3. Support Vector Regression -- 16.3.1. Regression Tube -- 16.3.2. Primal Form -- 16.3.3. Dual Form -- 16.4. Kernel Trick -- 16.5. Kernel Functions -- 16.5.1. Linear Kernel -- 16.5.2. Polynomial Kernel -- 16.5.3. Radial Kernel -- 16.5.4. Sigmoid Kernel. 16.6. Kernel Prediction -- 16.7. Kernel-based Algorithms -- 16.7.1. Kernel-based Svm -- 16.7.2. Kernel-based Svr -- 16.8. Conclusion -- 16.9. Further Readings -- References -- 17. Attribute Transformation -- 17.1. Introduction -- 17.2. Attribute Transformation Task -- 17.2.1. Target Task -- 17.2.2. Target Attribute -- 17.2.3. Transformed Attribute -- 17.2.4. Training Set -- 17.2.5. Modeling Transformations -- 17.2.6. Nonmodeling Transformations -- 17.3. Simple Transformations -- 17.3.1. Standardization -- 17.3.2. Normalization -- 17.3.3. Aggregation -- 17.3.4. Imputation -- 17.3.5. Binary Encoding -- 17.4. Multiclass Encoding -- 17.4.1. Encoding And Decoding Functions -- 17.4.2.1-ok-k Encoding -- 17.4.3. Error-correcting Encoding -- 17.4.4. Effects Of Multiclass Encoding -- 17.5. Conclusion -- 17.6. Further Readings -- References -- 18. Discretization -- 18.1. Introduction -- 18.2. Discretization Task -- 18.2.1. Motivation -- 18.2.2. Task Definition. 18.2.3. Discretization As Modeling -- 18.2.4. Discretization Quality -- 18.3. Unsupervised Discretization -- 18.3.1. Equal-width Intervals -- 18.3.2. Equal-frequency Intervals -- 18.3.3. Nonmodeling Discretization -- 18.4. Supervised Discretization -- 18.4.1. Pure-class Discretization -- 18.4.2. Bottom-up Discretization -- 18.4.3. Top-down Discretization -- 18.5. Effects Of Discretization -- 18.6. Conclusion -- 18.7. Further Readings -- References -- 19. Attribute Selection -- 19.1. Introduction -- 19.2. Attribute Selection Task -- 19.2.1. Motivation -- 19.2.2. Task Definition -- 19.2.3. Algorithms -- 19.3. Attribute Subset Search -- 19.3.1. Search Task -- 19.3.2. Initial State -- 19.3.3. Search Operators -- 19.3.4. State Selection -- 19.3.5. Stop Criteria -- 19.4. Attribute Selection Filters -- 19.4.1. Simple Statistical Filters -- 19.4.2. Correlation-based Filters -- 19.4.3. Consistency-based Filters -- 19.4.4. Relief -- 19.4.5. Random Forest -- 19.4.6. Cutoff Criteria. 19.4.7. Filter-driven Search -- 19.5. Attribute Selection Wrappers -- 19.5.1. Subset Evaluation -- 19.5.2. Wrapper Attribute Selection -- 19.6. Effects Of Attribute Selection -- 19.7. Conclusion -- 19.8. Further Readings -- References -- 20. Case Studies -- 20.1. Introduction -- 20.1.1. Datasets -- 20.1.2. Packages -- 20.1.3. Auxiliary Functions -- 20.2. Census Income -- 20.2.1. Data Loading And Preprocessing -- 20.2.2. Default Model -- 20.2.3. Incorporating Misclassification Costs -- 20.2.4. Pruning -- 20.2.5. Attribute Selection -- 20.2.6. Final Models -- 20.3.communities And Crime -- 20.3.1. Data Loading -- 20.3.2. Data Quality -- 20.3.3. Regression Trees -- 20.3.4. Linear Models -- 20.3.5. Attribute Selection -- 20.3.6. Piecewise-linear Models -- 20.4. Cover Type -- 20.4.1. Data Loading And Preprocessing -- 20.4.2. Class Imbalance -- 20.4.3. Decision Trees -- 20.4.4. Class Rebalancing -- 20.4.5. Multiclass Encoding -- 20.4.6. Final Classification Models -- 20.4.7. Clustering. 20.5. Conclusion -- 20.6. Further Readings -- References -- Closing -- A. Notation -- A.1. Attribute Values -- A.2. Data Subsets -- A.3. Probabilities -- B.r Packages -- B.1. Cran Packages -- B.2. Dmr Packages -- B.3. Installing Packages -- References -- C. Datasets. Pawel Cichosz. Includes Bibliographical References And Index. Mode Of Access: World Wide Web.