Active Learning COMS 6998-4: Learning and Empirical Inference

1. Active LearningCOMS 6998-4: Learning and Empirical Inference Irina Rish IBM T.J. Watson Research Center

2. Outline • Motivation • Active learning approaches • Membership queries • Uncertainty Sampling • Information-based loss functions • Uncertainty-Region Sampling • Query by committee • Applications • Active Collaborative Prediction • Active Bayes net learning

3. Standard supervised learning model Given m labeled points, want to learn a classifier with misclassification rate <, chosen from a hypothesis class H with VC dimension d < 1. VC theory: need m to be roughly d/, in the realizable case.

4. Active learning In many situations – like speech recognition and document retrieval – unlabeled data is easy to come by, but there is a charge for each label. What is the minimum number of labels needed to achieve the target error rate?

6. What is Active Learning? • Unlabeled data are readily available; labels are expensive • Want to use adaptive decisions to choose which labels to acquire for a given dataset • Goal is accurate classifier with minimal cost

7. Active learning warning • Choice of data is only as good as the model itself • Assume a linear model, then two data points are sufficient • What happens when data are not linear?

8. Active Learning Flavors Selective Sampling Membership Queries Pool Sequential Myopic Batch

9. Active Learning Approaches • Membership queries • Uncertainty Sampling • Information-based loss functions • Uncertainty-Region Sampling • Query by committee

12. Problem Many results in this framework, even for complicated hypothesis classes. [Baum and Lang, 1991] tried fitting a neural net to handwritten characters. Synthetic instances created were incomprehensible to humans! [Lewis and Gale, 1992] tried training text classifiers. “an artificial text created by a learning algorithm is unlikely to be a legitimate natural language expression, and probably would be uninterpretable by a human teacher.”

16. Uncertainty Sampling [Lewis & Gale, 1994] • Query the event that the current classifier is most uncertain about • Used trivially in SVMs, graphical models, etc. If uncertainty is measured in Euclidean distance: x x x x x x x x x x

17. Information-based Loss Function [MacKay, 1992] • Maximize KL-divergence between posterior and prior • Maximize reduction in model entropy between posterior and prior • Minimize cross-entropy between posterior and prior • All of these are notions of information gain

18. Query by Committee [Seung et al. 1992, Freund et al. 1997] • Prior distribution over hypotheses • Samples a set of classifiers from distribution • Queries an example based on the degree of disagreement between committee of classifiers x x x x x x x x x x C B A

19. Infogain-based Active Learning

20. Notation • We Have: • Dataset, D • Model parameter space, W • Query algorithm, q

21. Dataset (D) Example

22. Notation • We Have: • Dataset, D • Model parameter space, W • Query algorithm, q

23. Model Example St Ot Probabilistic Classifier Notation T : Number of examples Ot : Vector of features of example t St : Class of example t

24. Model Example Patient state (St) St : DiseaseState Patient Observations (Ot) Ot1 : Gender Ot2 : Age Ot3 : TestA Ot4 : TestB Ot5 : TestC

25. Gender TestB Age Gender S TestA S TestA Age TestB TestC TestC Possible Model Structures

26. Model Space St Ot Model: Model Parameters: P(St) P(Ot|St) Generative Model: Must be able to compute P(St=i, Ot=ot | w)

27. Model Parameter Space (W) • W = space of possible parameter values • Prior on parameters: • Posterior over models:

28. Notation • We Have: • Dataset, D • Model parameter space, W • Query algorithm, q q(W,D) returns t*, the next sample to label

29. Game while NotDone • Learn P(W | D) • q chooses next example to label • Expert adds label to D

30. Simulation ? ? S7=false S2=false S5=true ? O1 O2 O3 O4 O5 O6 O7 S1 S2 S3 S4 S5 S6 S7 hmm… q

31. Active Learning Flavors • Pool (“random access” to patients) • Sequential (must decide as patients walk in the door)

32. q? • Recall: q(W,D) returns the “most interesting” unlabelled example. • Well, what makes a doctor curious about a patient?

33. 1994

34. Score Function

35. Uncertainty Sampling Example FALSE

36. Uncertainty Sampling Example FALSE TRUE

37. Uncertainty Sampling GOOD: couldn’t be easier GOOD: often performs pretty well BAD: H(St) measures information gain about the samples, not the model Sensitive to noisy samples

38. Can we do better thanuncertainty sampling?

39. Informative with respect to what? 1992

40. Model Entropy P(W|D) P(W|D) P(W|D) W W W H(W) = high …better… H(W) = 0

41. Current model space entropy Expected model space entropy if we learn St Information-Gain • Choose the example that is expected to most reduce H(W) • I.e., Maximize H(W) – H(W | St)

42. Score Function

43. We usually can’t just sum over all models to get H(St|W) …but we can sample from P(W | D)

44. Conditional Model Entropy

45. Score Function

47. Score Function Familiar?

48. Uncertainty Sampling & Information Gain

49. But there is a problem…

50. If our objective is to reduce the prediction error, then “the expected information gain of an unlabeled sample is NOT a sufficient criterion for constructing good queries”