1 / 16

Chapter-8 -Instance Based Learning

Instance Based Learning

2715
Télécharger la présentation

Chapter-8 -Instance Based Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Instance Based Learning ?Read Ch? ?? ? k ?Nearest Neigh b or ? Lo cally w eigh ted regression ? Radial basis functions ? Case?based reasoning ? Lazy and eager learning c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  2. Instance?Based Learning Key idea? just store all training examples hx ? f ?x ?i i i Nearest neigh b or? ? Giv en query instance x ? ?rst lo ?x cate ? nearest q training example x ? then estimate n ? f ?x ? ? f ?x ? q n k ?Nearest neigh b or? ? Giv en x ? tak e v ote among its k nearest n brs ?if q discrete?v alued target function? ? tak e mean of f v alues of k nearest n brs ?if real?v alued? P k f i i?? ? f ?x ? ? q k c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  3. When T o Consider Nearest Neigh b or n ? Instances map to p oin ts in ? ? Less than ?? attributes p er instance ? Lots of training data Adv an tages? ? T raining is v ery fast ? Learn complex target functions ? Don?t lose information Disadv an tages? ? Slo w at query time ? Easily fo oled b y irrelev an t attributes c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  4. V oronoi Diagram − − − + + xq − + + − c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  5. Beha vior in the Limit Consider p?x? de?nes probabilit y that instance x will b e lab eled ? ?p ositiv e? v ersus ? ?negativ e?? Nearest neigh b or? ? As n um b er of training examples ? ?? approac hes Gibbs Algorithm Gibbs? with probabilit y p?x? predict ?? else ? k ?Nearest neigh b or? ? As n um b er of training examples ? ? and k gets large? approac hes Ba y es optimal Ba y es optimal? if p?x? ? ?? then predict ?? else ? Note Gibbs has at most t wice the exp ected error of Ba y es optimal c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  6. Distance?W eigh ted k NN Migh examples t w an t w eigh t of nearer just neigh b ors more hea vily ??? P k w f ?x ? i i i?? ? f ?x ? ? q P k w i i?? where ? w ? i ? d?x ? x ? q i and d?x ? x ? is distance b et w een x and x q i q i Note no w it mak es sense to use al l training instead k ? Shepard?s metho d c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  7. Curse of Dimensionali t y Imagine instances describ ed b y ?? attributes? but only ? are relev an t to target function Curse of dimensionality? nearest n br is easily mislead when high?dimensional X One approac h? ? Stretc h j th axis b y w eigh t z ? where z ? ? ? ? ? z j ? n c hosen to minimize prediction error ? Use cross?v alidati on to automatically c ho ose w eigh ts z ? ? ? ? ? z ? n ? Note setting z to zero eliminates this dimension j altogether see ?Mo ore and Lee? ????? c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  8. Lo cally W eigh ted Regression Note k NN forms lo cal appro ximation to f for eac h query p oin t x q ? Wh y not form an explici t appro ximation f ?x? for region surrounding x q ? Fit linear function to k nearest neigh b ors ? Fit quadratic? ??? ? Pro duces ?piecewise appro ximation? to f Sev eral c hoices of error to minimize? ? Squared error o v er k nearest neigh b ors ? X ? ? E ?x ? ? ?f ?x? ? f ?x?? ? q ? x? k near est nbr s of x q ? Distance?w eigh ted squared error o v er all n brs ? X ? ? E ?x ? ? ?f ?x? ? f ?x?? K ?d?x ? x?? ? q q ? x?D ? ? ? ? c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  9. Radial Basis F unction Net w orks ? Global appro ximation to target function? in terms of linear com bination of lo cal appro ximations ? Used? e?g?? for image classi?cati on ? A di?eren t kind of neural net w ork ? Closely related to distance?w eigh ted regression? but ?eager? instead of ?lazy? c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  10. Radial Basis F unction Net w orks f(x) where a ?x? are the attributes describing instance i x? and k X f ?x? ? w ? w K ?d?x ? x?? w0 1 w1 wk ? u u u u?? ... One common c hoice for K ?d?x ? x?? is u u ? ? ? d ?x ?x? u ? ?? u K ?d?x ? x?? ? e u u ... a (x) a (x) a (x) 1 2 n c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  11. T raining Radial Basis F unction Net? w orks Q?? What x to use for eac h k ernel function u K ?d?x ? x?? u u ? Scatter uniformly throughout instance space ? Or use training instances ?re?ects instance distribution? Q?? Ho w to train w eigh ts ?assume here Gaussian K ? u ? First c ho ose v ariance ?and p erhaps mean? for eac h K u ? e?g?? use EM ? Then hold K ?xed? and train linear output la y er u ? e?cien t metho ds to ?t linear function c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  12. Case?Based Reasoning Can apply instance?based learning ev en when n X ?? ? ? need di?eren t ?distance? metric Case?Based Reasoning is instance?based learning applied to instances with sym b olic logic descriptions ??user?complaint error???on?shutd own? ?cpu?model PowerPC? ?operating?system Windows? ?network?connecti on PCIA? ?memory ??meg? ?installed?applic ation s Excel Netscape VirusScan? ?disk ?gig? ?likely?cause ????? c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  13. Case?Based Reasoning in CADET CADET? ?? stored examples of mec hanical devices ? eac h training example? h qualitati v e function? mec hanical structurei ? new query? desired function? ? target v alue? mec hanical structure for this function Distance metric? matc h qualitat i v e function descriptions c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  14. Case?Based Reasoning in CADET A stored case: T−junction pipe Structure: Function: Q ,T 1 T Q = temperature = waterflow Q 1 + 1 Q 3 Q + 2 Q ,T 3 3 T + 1 T 3 Q ,T2 T + 2 2 A problem specification: Water faucet Structure: Function: + ? C Q + + t c + + Q c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ???? m C Q + − f h + + T c T m T + h

  15. Case?Based Reasoning in CADET ? Instances represen ted b y ric h structural descriptions ? Multiple cases retriev ed ?and com bined? to form solution to new problem ? Tigh t coupling b et w een case retriev al and problem solving Bottom line? ? Simple matc hing of cases useful for tasks suc h as answ ering help?desk queries ? Area of ongoing researc h c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

  16. Lazy and Eager Learning Lazy? w ait for query b efore generalizi ng ? k ?Nearest Neighbor? Case based reasoning Eager? generalize b efore seeing query ? Radial basis function net w orks? ID?? Bac kpropagation? Naiv eBa y es? ? ? ? Do es it matter? ? Eager learner m ust create global appro ximation ? Lazy learner can create man y lo cal appro ximations ? if they use same H ? lazy can represen t more complex fns ?e?g?? consider H ? linear functions? c ??? lecture slides for textb o ok Machine L e arning? ? T om M? Mitc hell? McGra w Hill? ????

More Related