U.S. Department of the Interior U.S. Geological Survey

1. Decision Trees for Land Cover Mapping Guilty Parties: B. Wylie, C. Homer, C. Huang, L. Yang, M. Coan EROS Data Center, Sioux Falls U.S. Department of the Interior U.S. Geological Survey

2. Tab contents + Concept + Advantages + Training + Descriptive and Predictive Trees + More than Just Class Prediction + Cross Validation + Splus versus See5/C5 + Hierarchical Trees + Recipe for Success

3. How do you eat an elephant? One bite at time! Divide & Conquer Stratify & Predict

4. Separating Apples and Oranges (Lemons?) 6/7 = 86% Weight 6/6 = 100% Specific Gravity

5. Example Decision Tree True False Mnf1<=28 Mnf3<=19 Mnf13>56 decid. Mnf17>56 Mnf16<=54 Mnf1>19 Mnf3<=24 decid. shrub P. pine cedar Mnf1<=25 Mnf1<=28 Mnf3<=38 Mnf8<=28 Mnf11<51 cedar Mnf2<=43 decid. shrub shrub cedar cedar P. pine

6. Advantages of Decision Trees • Rapid • Repeatable • Nonparametric • Utilize categorical data • Non-linear relationships • Less sensitive to errors in training data Disadvantages of Decision Trees • Lots of training • Over-fitting • Weights toward the relative % of training data • Short-sighted (stepwise methods fail to identify optimal subset regressors –Fox,1991)

7. Other Methods: Unsupervised Clustering + Cluster busting can be time consuming + Cluster interpretation is subjective + Cannot include categorical inputs + Difficult to interpret if multiple date or if non spectral data include (DEM) + Parametric (assumes normal distribution) Supervised Classification + Parametric (assumes normal distribution) + Cannot include categorical inputs + Problematic multiple date or if non-spectral data include (DEM) + Difficult for large area applications Neural Nets + Long convergence times (training) + High CPU demands (training) + Grey box + Tricked by local minimums + Non-repeatability of results (random search functions) + Sensitive to errors in training data

8. Training data: capture variability of a class (sample size) Spectral variability: A monoculture of wheat

9. Would 2 examples of each produce a reliable separation? Weight Specific Gravity

10. Training samples Classification tree is a “Data Mining” method so it performs well with large training data sets. Sampling of classes should reflect their relative frequency in the study area. rare classes = few training points common classes = many training points Adequate but not over sampling of rare classes Samples should be widely distributed over the study area to minimize autocorrelation effects and allow effective use of date band information.

11. Descriptive or Prediction decision trees? (De’ath and Fabricus 2000) DESCRIPTIVE TREE: 1) A single tree 2) Objective is to understand important factors or functional relationships 3) The decisions used by the tree are as important as the predictions • Drivers of bear and deer habitat (Kobler and Adamic 2000, Debeljak et al. 2001) • Predicting species distributions (Vayssieres, et al. 2000) • + CART outperformed logistic regression

12. PREDICTION TREES: • Objective is Best Possible Predictions • Combination of multiple trees • Higher accuracies, more stable and • robust (DeFries and Chan 2000)

13. Multiple Tree Approaches: Prediction • Bagging (bootstrap sampling of training data)--Splus & C5 • Subset data layers—Splus & C5 • Boosting* – C5

14. Multiple Tree Approaches: Prediction 2) Subset of data layers—Splus & C5 Tree 1 V O T E soils Tree 2 spectral Tree 3 LUDA

15. Multiple Tree Approaches V O T E 2) Boosting (iterative tree’s try to account for previous tree’s errors)—C5 Different over-fitting issues associated with each tree tend to be averaged out.

16. Single tree Boosted

17. Boosting versus Single Tree (Zone 16, Independent Test Data)

18. Trees provide more than just land cover predictions Mnf1<=28 Mnf3<=19 Mnf13>56 decid. Mnf17>56 Mnf16<=54 Mnf1>19 Mnf3<=24 decid. shrub P. pine cedar Mnf1<=25 Mnf1<=28 Mnf3<=38 Mnf8<=28 Mnf11<51 cedar Mnf2<=43 decid. shrub shrub cedar cedar P. pine At each “terminal node” or “leaf” we: + know the number of training points correct, incorrect, and % right + could assign arbitrary “node numbers”

19. % right or confidence “leaf” map land cover

20. Identification of “useful” tree inputs Use relative frequency of use of data layers in the training data as a crude index of data layer “utility”. Top “utility” data layers from 40 possible input layers

21. How to use “utility” of input data layers • Reduce inputs to decision tree • Reduced tree may have improved accuracies • Increases speed that the tree can be applied to the study area • Interpretation of underlying functional relationships (drivers) • Produce multiple trees for class “voting”

22. Honest Real-time Error Assessment : Cross Validation (3 fold)

23. Accuracy Assessment: Cross Validation versus Independent Test, Zone 16, Utah

24. Uses of Cross Validation • Accuracy Assessment • Optimal tree data sets • Pruning • All training data used for prediction Cautions • spatial autocorrelation • look for “significant” error changes when pruning or selecting tree parameters

25. Splus – See5/C5 Bake Off

26. Past Experiences: hierarchical implementation of trees Landsat 7 ETM+ Mosaic (band 5,4,3) Mapping Zone 60, Spring, 2000 and 2001

27. Forest and Non-Forest Classification Mapping Zone 60, 2001 Established a classification tree model for mapping forest and non-forest class using 1700+ FIA plot data (669 forest and 1100+ non-forest plots). The classification was run using a 5-fold cross-validation procedure. The agreement between mapped and reference/validation data is 95% with standard error (SE) less than 1.0%.

28. Forest Classification Based on NLCD 2000 Classification System Mapping Zone 60, 2001 Established a classification tree model for mapping three MRLC forest classes using 669 FIA plots and 5-fold cross-validation procedure (134 plots for validation for each of the 5 runs). The agreement between mapped and reference/validation data is 80% with SE 1.0%

29. Forest Type Group Classification Based on USFS classification System Mapping Zone 60, 2001 Established a classification tree model for mapping six forest type groups using 669 FIA plots and a 5-fold cross-validation procedure (134 plots for validation for each of the 5 runs). The agreement between mapped and reference/validation data is 65% with SE 2.3%

30. Leaf-on Landsat 7 ETM+ scene mosaic (bands 5,4,3) for mapping zone 16 – Utah/Southern Idaho

31. Forest/non-forest classification for mapping zone 16 – Utah/Southern Idaho 0 * 16.8% 1 * 18.8% 2 * 19.1% 3 * 19.4% 4 * 16.3% Mean 18.1% SE 0.6%

32. Deciduous/evergreen/ mixed classification for mapping zone 16 – Utah/Southern Idaho Fold Decision Tree ---- ---------------- Size Errors 0 * 19.7% 1 * 19.9% 2 * 18.5% 3 * 19.4% 4 * 20.8% Mean 19.7% SE 0.4%

33. Forest type group classification for mapping zone 16 – Utah/Southern Idaho Fold Decision Tree ---- ---------------- Size Errors 0 * 31.9% 1 * 37.3% 2 * 36.8% 3 * 40.0% 4 * 31.9% 5 * 37.8% 6 * 33.0% 7 * 35.7% 8 * 35.5% 9 * 32.3% Mean 35.2% SE 0.9%

34. Mean and standard error (in parenthesis) of the overall accuracy (%) of classifications developed using a descriptive tree and an hierarchical tree approach in 5 repeated experiments. (Zone 16)

35. Recipe for Success • Adequate and representative training (adequately represent rare classes, preserve relative proportions of training and population) • Model construction assessed with Cross Validation • (boosting, pruning, and data layer exclusions) • Multiple trees for mapping (boosting) • Visual inspect land cover and add training in miss-classified areas and reconstruct model • Consider “hierarchical” trees to allow trees to focus on problematic separations • Avoid See5/C5 “rule set” option (SLOW when applied spatially)

36. Training Data Collection From Imagine Files Geo-registration linking? Ignore values! X,Y, Land cover (*.dat)?

38. Apply Tree Spatially Avoid Rules!! Mask!!!!! Confidence map!!