1 / 3

Decision Trees

This document explores decision trees using an example from a survey aimed at determining customer preferences for a new car model. It highlights how to select customers for a training set for an advertising campaign by computing the information gain and gain ratios based on attributes like age, city, and car preference. Key computations are demonstrated using entropy functions and gain formulas. The methodology aligns with ID3 and C5.0 algorithms to assess the quality of attributes in classification problems, showing how to derive insights from customer data effectively.

terrell
Télécharger la présentation

Decision Trees

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Decision Trees • Example: • Conducted survey to see what customers were interested in new model car • Want to select customers for advertising campaign training set

  2. Basic Information Gain Computations Result: I_Gain_Ratio: city>age>car Result: I_Gain: age > car=city Gain(D,city=)= H(1/3,2/3) – ½ H(1,0) – ½ H(1/3,2/3)=0.45 D=(2/3,1/3) G_Ratio_pen(city=)=H(1/2,1/2)=1 city=la city=sf D1(1,0) D2(1/3,2/3) Gain(D,car=)= H(1/3,2/3) – 1/6 H(0,1) – ½ H(2/3,1/3) – 1/3 H(1,0)=0.45 D=(2/3,1/3) G_Ratio_pen(car=)=H(1/2,1/3,1/6)=1.45 car=van car=merc car=taurus D3(1,0) D2(2/3,1/3) D1(0,1) Gain(D,age=)= H(1/3,2/3) – 6*1/6 H(0,1) = 0.90 G_Ratio_pen(age=)=log2(6)=2.58 D=(2/3,1/3) age=22 age=25 age=27 age=35 age=40 age=50 D1(1,0) D3(1,0) D4(1,0) D5(1,0) D2(0,1) D6(0,1)

  3. C5.0/ID3 Test Selection • Assume we have m classes in our classification problem. A test S subdivides the examples D= (p1,…,pm) into n subsets D1 =(p11,…,p1m) ,…,Dn =(p11,…,p1m). The qualify of S is evaluated using Gain(D,S) (ID3) or GainRatio(D,S) (C5.0): • Let H(D=(p1,…,pm))= Si=1 (pi log2(1/pi)) (called the entropy function) • Gain(D,S)= H(D) -Si=1 (|Di|/|D|)*H(Di) Gain_Ratio(D,S)= Gain(D,S) / H(|D1|/|D|,…, |Dn|/|D|) Remarks: • |D| denotes the number of elements in set D. • D=(p1,…,pm) implies that p1+…+ pm =1 and indicates that of the |D| examples p1*|D| examples belong to the first class, p2*|D| examples belong to the second class,…, and pm*|D| belong the m-th (last) class. • H(0,1)=H(1,0)=0; H(1/2,1/2)=1, H(1/4,1/4,1/4,1/4)=2, H(1/p,…,1/p)=log2(p). • C5.0 selects the test S with the highest value for Gain_Ratio(D,S), whereas ID3 picks the test S for the examples in set D with the highest value for Gain (D,S). m n

More Related