Data Mining with Decision Trees : Theory and Applications.
Title:
Data Mining with Decision Trees : Theory and Applications.
Author:
Rokach, Lior.
ISBN:
9789812771728
Personal Author:
Physical Description:
1 online resource (263 pages)
Series:
Series in Machine Perception and Artificial Intelligence ; v.69
Series in Machine Perception and Artificial Intelligence
Contents:
Contents -- Preface -- 1. Introduction to Decision Trees -- 1.1 Data Mining and Knowledge Discovery -- 1.2 Taxonomy of Data Mining Methods -- 1.3 Supervised Methods -- 1.3.1 Overview -- 1.4 Classification Trees -- 1.5 Characteristics of Classification Trees -- 1.5.1 Tree Size -- 1.5.2 The hierarchical nature of decision trees -- 1.6 Relation to Rule Induction -- 2. Growing Decision Trees -- 2.0.1 Training Set -- 2.0.2 Definition of the Classification Problem -- 2.0.3 Induction Algorithms -- 2.0.4 Probability Estimation in Decision Trees -- 2.0.4.1 Laplace Correction -- 2.0.4.2 No Match -- 2.1 Algorithmic Framework for Decision Trees -- 2.2 Stopping Criteria -- 3. Evaluation of Classification Trees -- 3.1 Overview -- 3.2 Generalization Error -- 3.2.1 Theoretical Estimation of Generalization Error -- 3.2.2 Empirical Estimation of Generalization Error -- 3.2.3 Alternatives to the Accuracy Measure -- 3.2.4 The F-Measure -- 3.2.5 Confusion Matrix -- 3.2.6 Classifier Evaluation under Limited Resources -- 3.2.6.1 ROC Curves -- 3.2.6.2 Hit Rate Curve -- 3.2.6.3 Qrecall (Quota Recall) -- 3.2.6.4 Lift Curve -- 3.2.6.5 Pearson Correlation Coegfficient -- 3.2.6.6 Area Under Curve (AUC) -- 3.2.6.7 Average Hit Rate -- 3.2.6.8 Average Qrecall -- 3.2.6.9 Potential Extract Measure (PEM) -- 3.2.7 Which Decision Tree Classifier is Better? -- 3.2.7.1 McNemar's Test -- 3.2.7.2 A Test for the Difference of Two Proportions -- 3.2.7.3 The Resampled Paired t Test -- 3.2.7.4 The k-fold Cross-validated Paired t Test -- 3.3 Computational Complexity -- 3.4 Comprehensibility -- 3.5 Scalability to Large Datasets -- 3.6 Robustness -- 3.7 Stability -- 3.8 Interestingness Measures -- 3.9 Overfitting and Underfitting -- 3.10 "No Free Lunch" Theorem -- 4. Splitting Criteria -- 4.1 Univariate Splitting Criteria -- 4.1.1 Overview -- 4.1.2 Impurity based Criteria.
4.1.3 Information Gain -- 4.1.4 Gini Index -- 4.1.5 Likelihood Ratio Chi-squared Statistics -- 4.1.6 DKM Criterion -- 4.1.7 Normalized Impurity-based Criteria -- 4.1.8 Gain Ratio -- 4.1.9 Distance Measure -- 4.1.10 Binary Criteria -- 4.1.11 Twoing Criterion -- 4.1.12 Orthogonal Criterion -- 4.1.13 Kolmogorov-Smirnov Criterion -- 4.1.14 AUC Splitting Criteria -- 4.1.15 Other Univariate Splitting Criteria -- 4.1.16 Comparison of Univariate Splitting Criteria -- 4.2 Handling Missing Values -- 5. Pruning Trees -- 5.1 Stopping Criteria -- 5.2 Heuristic Pruning -- 5.2.1 Overview -- 5.2.2 Cost Complexity Pruning -- 5.2.3 Reduced Error Pruning -- 5.2.4 Minimum Error Pruning (MEP) -- 5.2.5 Pessimistic Pruning -- 5.2.6 Error-Based Pruning (EBP) -- 5.2.7 Minimum Description Length (MDL) Pruning -- 5.2.8 Other Pruning Methods -- 5.2.9 Comparison of Pruning Methods -- 5.3 Optimal Pruning -- 6. Advanced Decision Trees -- 6.1 Survey of Common Algorithms for Decision Tree Induction -- 6.1.1 ID3 -- 6.1.2 C4.5 -- 6.1.3 CART -- 6.1.4 CHAID -- 6.1.5 QUEST. -- 6.1.6 Reference to Other Algorithms -- 6.1.7 Advantages and Disadvantages of Decision Trees -- 6.1.8 Oblivious Decision Trees -- 6.1.9 Decision Trees Inducers for Large Datasets -- 6.1.10 Online Adaptive Decision Trees -- 6.1.11 Lazy Tree -- 6.1.12 Option Tree -- 6.2 Lookahead -- 6.3 Oblique Decision Trees -- 7. Decision Forests -- 7.1 Overview -- 7.2 Introduction -- 7.3 CombinationMethods -- 7.3.1 WeightingMethods -- 7.3.1.1 Majority Voting -- 7.3.1.2 Performance Weighting -- 7.3.1.3 Distribution Summation -- 7.3.1.4 Bayesian Combination -- 7.3.1.5 Dempster-Shafer -- 7.3.1.6 Vogging -- 7.3.1.7 Naı̈ve Bayes -- 7.3.1.8 Entropy Weighting -- 7.3.1.9 Density-based Weighting -- 7.3.1.10 DEA Weighting Method -- 7.3.1.11 Logarithmic Opinion Pool -- 7.3.1.12 Gating Network -- 7.3.1.13 Order Statistics.
7.3.2 Meta-combination Methods -- 7.3.2.1 Stacking -- 7.3.2.2 Arbiter Trees -- 7.3.2.3 Combiner Trees -- 7.3.2.4 Grading -- 7.4 Classifier Dependency -- 7.4.1 Dependent Methods -- 7.4.1.1 Model-guided Instance Selection -- 7.4.1.2 Incremental Batch Learning -- 7.4.2 Independent Methods -- 7.4.2.1 Bagging -- 7.4.2.2 Wagging -- 7.4.2.3 Random Forest -- 7.4.2.4 Cross-validated Committees -- 7.5 Ensemble Diversity -- 7.5.1 Manipulating the Inducer -- 7.5.1.1 Manipulation of the Inducer's Parameters -- 7.5.1.2 Starting Point in Hypothesis Space -- 7.5.1.3 Hypothesis Space Traversal -- 7.5.2 Manipulating the Training Samples -- 7.5.2.1 Resampling -- 7.5.2.2 Creation -- 7.5.2.3 Partitioning -- 7.5.3 Manipulating the Target Attribute Representation -- 7.5.4 Partitioning the Search Space -- 7.5.4.1 Divide and Conquer -- 7.5.4.2 Feature Subset-based Ensemble Methods -- 7.5.5 Multi-Inducers -- 7.5.6 Measuring the Diversity -- 7.6 Ensemble Size -- 7.6.1 Selecting the Ensemble Size -- 7.6.2 Pre Selection of the Ensemble Size -- 7.6.3 Selection of the Ensemble Size while Training -- 7.6.4 Pruning - Post Selection of the Ensemble Size -- 7.6.4.1 Pre-combining Pruning -- 7.6.4.2 Post-combining Pruning -- 7.7 Cross-Inducer -- 7.8 Multistrategy Ensemble Learning -- 7.9 Which Ensemble Method Should be Used? -- 7.10 Open Source for Decision Trees Forests -- 8. Incremental Learning of Decision Trees -- 8.1 Overview -- 8.2 The Motives for Incremental Learning -- 8.3 The Inefficiency Challenge -- 8.4 The Concept Drift Challenge -- 9. Feature Selection -- 9.1 Overview -- 9.2 The "Curse of Dimensionality" -- 9.3 Techniques for Feature Selection -- 9.3.1 Feature Filters -- 9.3.1.1 FOCUS -- 9.3.1.2 LVF. -- 9.3.1.3 Using One Learning Algorithm as a Filter for Another -- 9.3.1.4 An Information Theoretic Feature Filter.
9.3.1.5 An Instance Based Approach to Feature Selection - RELIEF -- 9.3.1.6 Simba and G-flip -- 9.3.1.7 Contextual Merit Algorithm -- 9.3.2 Using Traditional Statistics for Filtering -- 9.3.2.1 Mallows Cp -- 9.3.2.2 AIC, BIC and F-ratio -- 9.3.2.3 Principal Component Analysis (PCA) -- 9.3.2.4 Factor Analysis (FA) -- 9.3.2.5 Projection Pursuit -- 9.3.3 Wrappers -- 9.3.3.1 Wrappers for Decision Tree Learners -- 9.4 Feature Selection as a Means of Creating Ensembles -- 9.5 Ensemble Methodology as a Means for Improving Feature Selection -- 9.5.1 Independent Algorithmic Framework -- 9.5.2 Combining Procedure -- 9.5.2.1 Simple Weighted Voting -- 9.5.2.2 Naı̈ve Bayes Weighting using Arti.cial Contrasts -- 9.5.3 Feature EnsembleGenerator -- 9.5.3.1 Multiple Feature Selectors -- 9.5.3.2 Bagging -- 9.6 Using Decision Trees for Feature Selection -- 9.7 Limitation of Feature Selection Methods -- 10. Fuzzy Decision Trees -- 10.1 Overview -- 10.2 Membership Function -- 10.3 Fuzzy Classification Problems -- 10.4 Fuzzy Set Operations -- 10.5 Fuzzy Classification Rules -- 10.6 Creating Fuzzy Decision Tree -- 10.6.1 Fuzzifying Numeric Attributes -- 10.6.2 Inducing of Fuzzy Decision Tree -- 10.7 Simplifying the Decision Tree -- 10.8 Classification of New Instances -- 10.9 Other Fuzzy Decision Tree Inducers -- 11. Hybridization of Decision Trees with other Techniques -- 11.1 Introduction -- 11.2 A Decision Tree Framework for Instance-Space Decomposition -- 11.2.1 Stopping Rules -- 11.2.2 Splitting Rules -- 11.2.3 Split Validation Examinations -- 11.3 The CPOMAlgorithm -- 11.3.1 CPOM Outline -- 11.3.2 The Grouped Gain Ratio Splitting Rule -- 11.4 Induction of Decision Trees by an Evolutionary Algorithm -- 12. Sequence Classification Using Decision Trees -- 12.1 Introduction -- 12.2 Sequence Representation -- 12.3 Pattern Discovery -- 12.4 Pattern Selection.
12.4.1 Heuristics for Pattern Selection -- 12.4.2 Correlation based Feature Selection -- 12.5 Classifier Training -- 12.5.1 Adjustment of Decision Trees -- 12.5.2 Cascading Decision Trees -- 12.6 Application of CREDT in Improving of Information Retrieval ofMedical Narrative Reports -- 12.6.1 Related Works -- 12.6.1.1 Text Classification -- 12.6.1.2 Part-of-speech Tagging -- 12.6.1.3 Frameworks for Information Extraction -- 12.6.1.4 Frameworks for Labeling Sequential Data -- 12.6.1.5 Identifying Negative Context in Nondomain Specific Text (General NLP) -- 12.6.1.6 Identifying Negative Context in Medical Narratives -- 12.6.1.7 Works Based on Knowledge Engineering -- 12.6.1.8 Works based on Machine Learning -- 12.6.2 Using CREDT for Solving the Negation Problem -- 12.6.2.1 The Process Overview -- 12.6.2.2 Step 1: Corpus Preparation -- 12.6.2.3 Step 1.1: Tagging -- 12.6.2.4 Step 1.2: Sentence Boundaries -- 12.6.2.5 Step 1.3: Manual Labeling -- 12.6.2.6 Step 2: Patterns Creation -- 12.6.2.7 Step 3: Patterns Selection -- 12.6.2.8 Step 4: Classi.er Training -- 12.6.2.9 Cascade of Three Classifiers -- Bibliography -- Index.
Abstract:
This is the first comprehensive book dedicated entirely to the field of decision trees in data mining and covers all aspects of this important technique. Decision trees have become one of the most powerful and popular approaches in knowledge discovery and data mining, the science and technology of exploring large and complex bodies of data in order to discover useful patterns. The area is of great importance because it enables modeling and knowledge extraction from the abundance of data available. Both theoreticians and practitioners are continually seeking techniques to make the process more efficient, cost-effective and accurate. Decision trees, originally implemented in decision theory and statistics, are highly effective tools in other areas such as data mining, text mining, information extraction, machine learning, and pattern recognition. This book invites readers to explore the many benefits in data mining that decision trees offer: -->: Self-explanatory and easy to follow when compacted; Able to handle a variety of input data: nominal, numeric and textual; Able to process datasets that may have errors or missing values; High predictive performance for a relatively small computational effort; Available in many data mining packages over a variety of platforms; Useful for various tasks, such as classification, regression, clustering and feature selection -->. Sample Chapter(s). Chapter 1: Introduction to Decision Trees (245 KB). Chapter 6: Advanced Decision Trees (409 KB). Chapter 10: Fuzzy Decision Trees (220 KB). Contents: Introduction to Decision Trees; Growing Decision Trees; Evaluation of Classification Trees; Splitting Criteria; Pruning Trees; Advanced Decision Trees; Decision Forests; Incremental Learning of Decision Trees; Feature Selection; Fuzzy Decision Trees; Hybridization of Decision Trees with Other Techniques; Sequence
Classification Using Decision Trees. Readership: Researchers, graduate and undergraduate students in information systems, engineering, computer science, statistics and management.
Local Note:
Electronic reproduction. Ann Arbor, Michigan : ProQuest Ebook Central, 2017. Available via World Wide Web. Access may be limited to ProQuest Ebook Central affiliated libraries.
Genre:
Added Author:
Electronic Access:
Click to View