1 / 20

Lazy Learning k -Nearest Neighbour

Lazy Learning k -Nearest Neighbour. Motivation: availability of large amounts of processing power improves our ability to tune k -NN classifiers. What is Lazy Learning?. Compare ANNs and CBR or k -NN classifier Artificial Neural Networks are eager learners

skah
Télécharger la présentation

Lazy Learning k -Nearest Neighbour

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lazy Learningk-Nearest Neighbour Motivation: availability of large amounts of processing power improves our ability to tunek-NN classifiers

  2. What is Lazy Learning? • Compare ANNs and CBR or k-NN classifier • Artificial Neural Networks are eager learners • training examples compiled into a model at training time • not available at runtime • CBR or k-Nearest Neighbour are lazy • little offline learning done • work deferred to runtime Compare conventional use of lazy-eager in computer science

  3. Outline • Classification problems • Classification techniques • k-Nearest Neighbour • Condense training Set • Feature selection • Feature weighting • Ensemble techniques in ML

  4. Classification problems • Exemplar characterised by a set of features; decide class to which exemplar belongs Compare regression problems • Exemplar characterised by a set of features; • decide value of continuous output (dependant) variable

  5. Classifying apples and pears To what class does this belong?

  6. Distance/Similarity Function For query q and training set X (described by features F) compute d(x,q) for each x  X, where and where Category of q decided by its k Nearest Neighbours

  7. k-NN and Noise • 1-NN easy to implement • susceptible to noise • a misclassification every time a noisy pattern retrieved • k-NN with k  3 will overcome this

  8. e.g. Pregnancy prediction http://svr-www.eng.cam.ac.uk/projects/qamc/

  9. e.g. MVT • components present or absent • solder joints good or bad • Machine Vision for inspection of PCBs

  10. Components present? Absent Present

  11. Characterise image as a set of features

  12. Classification techniques • Artificial Neural Networks • also good for non linear regression • black box • development tricky • users do not know what is going on • Decision Trees • built using induction (information theoretic analysis) • k-Nearest Neighbour classifiers • keep training examples, find k nearest at run time

  13. Dimension reduction in k-NN Feature Selection q best features • Not all features required • noisy features a hindrance • Some examples redundant • retrieval time depends on no. of examples n covering examples Condensed NN m examples p features

  14. Condensed NN D set of training samples Find E where E D; NN rule used with E should be as good as with D choose x  D randomly, D  D \ {x}, E  {x}, DO learning?  FALSE, FOR EACH x  D classify x by NN using E, if classification incorrect then E  E  {x}, D  D \ {x}, learning  TRUE, WHILE (learning?  FALSE)

  15. 100 examples 2 categories Different CNN solutions Condensed NN

  16. identify exemplars near decision surface • in diagram B more useful than A A B Improving Condensed NN • Different outcomes depending on data order • that’s a bad thing in an algorithm • Sort data based on distance to nearest unlike neighbour

  17. CNN using NUN 100 examples 2 categories Different CNN solutions Condensed NN

  18. Feature selection • Irrelevant features are noise: • make classification harder • Extra features add to computation cost p m

  19. Outcome Combiner Classifiers Ensemble techniques • For the user with more machine cycles than they know what to do with • Build several classifiers • different training subsets • different feature subsets • Aggregate results • voting • vote based on generalisation error

  20. Conclusions • Finding a covering set of training data • very good solutions exist • Compare with results of Ensemble techniques

More Related