Learning From Observations

1. Learning From Observations “In which we describe agents that can improve their behavior through diligent study of their own experiences.” -Artificial Intelligence: A Modern Approach

2. What is Machine Learning Machine Learning is the domain of Artificial Intelligence which is concerned with building adaptive computer systems that are able to improve their competence and/or efficiency through learning from input data or from their own problem solving experience.

3. The architecture of a learning agent Implements a general problem solving method that uses the knowledge from the knowledge base to interpret the input and provide an appropriate output. Implements learning methods for extending and refining the knowledge base to improve agent’s competence and/or efficiency in problem solving. Learning Agent Input/ Problem Solving Engine Sensors Learning Engine User/ Environment Output/ Ontology Rules/Cases/Methods Knowledge Base Effectors Data structures that represent the objects from the application domain, general laws governing them, actions that can be performed with them, etc.

4. Learning Agent 2 pendekatan membangun agent: • Dirancang, diprogram, diberi knowledge oleh manusia • Dirancang sehingga bisa belajar dari input (percept, pengalaman, dst.) Manfaat Learning Agent: • Environment bisa berubah • Manuasi (programmer?) itu malas, ceroboh, tidak maha tahu

5. Agent yang belajar Kita sudah melihat banyak jenis agent: • Simple reflex agent (condition-action rules) • Search agent (punya goal dan successor state function) • Knowledge-based agent (membangun representasi simbolik dari percept) • Utility-based agent (mengukur nilai utility sebuh state) Semua agent ini bisa dibangun dengan metode “pembelajaran” yang tepat! Contoh: agent supir taksi. Bagaimana dia belajar?

6. Arsitektur Learning Agent

7. Jenis Metode Learning • Supervised Learning • Agent belajar fungsi yang memetakan input ke output • Pada tahap training, learning algorithm menerima sekumpulan input beserta output yang diharapkan. • Sample ini dipakai untuk estimasi fungsinya • Unsupervised Learning • Sebuah learning algorithm menerima sekumpulan data, dan harus menemukan pola-pola di dalamanya. • Misalnya: • Sebuah agent taxi menerima data mengenai laju lalin sepanjang hari. Mungkin ia bisa belajar periode “morning rush hour”, “evening rush hour”

8. Jenis Metode Learning • Reinforcement Learning • Sebuah agent menerima input data dan harus mengambil tindakan. • Agent lalu menerima reinforcement signal (mis. Good, bad) sebagai akibat tindakan. • Learning algorithm memodifikasi agent function untuk memaksimalkan signal “good”.

9. Inductive Learning

10. Consistency & Simplicity

11. Contoh “Curve-fitting”

12. Contoh “Curve-fitting”

13. Contoh “Curve-fitting”

14. Contoh “Curve-fitting”

15. Contoh “Curve-fitting”

16. Metode Ilmiah = Inductive Learning

17. Induction sebagai search

18. Decision Tree

19. Sample Decision Tree http://www-lmmb.ncifcrf.gov/~toms/paper/primer/latex/index.html http://directory.google.com/Top/Science/Math/Applications/Information_Theory/Papers/

20. Information Theory

21. INFORMATION THEORY - ENTROPY • Entropy at a given node t: (NOTE: p( j | t) is the relative frequency of class j at node t). • Measures homogeneity of a node. • Maximum (log nc) when records are equally distributed among all classes implying least information • Minimum (0.0) when all records belong to one class, implying most information

22. Examples for computing Entropy P(C1) = 0/6 = 0 P(C2) = 6/6 = 1 Entropy = – 0 log 0– 1 log 1 = – 0 – 0 = 0 P(C1) = 1/6 P(C2) = 5/6 Entropy = – (1/6) log2 (1/6)– (5/6) log2 (5/6) = 0.65 P(C1) = 2/6 P(C2) = 4/6 Entropy = – (2/6) log2 (2/6)– (4/6) log2 (4/6) = 0.92

23. Entropy: Used by ID3 Entropy(S) = - p log2 p - q log2 q • Entropy measures the impurity of S • S is a set of examples • p is the proportion of positive examples • q is the proportion of negative examples

24. ENTROPY – Information Gain • Information Gain: Parent Node, p is split into k partitions; niis number of records in partition I

25. ID3 pno = 5/14 play pyes = 9/14 don’t play Impurity= - pyes log2 pyes - pno log2 pno • = - 9/14 log2 9/14 - 5/14 log2 5/14 • = 0.94 bits

26. ID3 outlook humidity temperature windy sunny overcast rainy high normal hot mild cool false true play 0.98 bits * 7/14 0.92 bits * 6/14 0.81 bits * 4/14 0.81 bits * 8/14 0.0 bits * 4/14 0.97 bits * 5/14 0.59 bits * 7/14 1.0 bits * 4/14 1.0 bits * 6/14 don’t play + = 0.69 bits + = 0.79 bits + = 0.91 bits + = 0.89 bits gain: 0.15 bits gain: 0.03 bits gain: 0.05 bits 0.94 bits maximal information gain amount of information required to specify class of an example given that it reaches node 0.97 bits * 5/14 gain: 0.25 bits

27. humidity temperature windy high normal hot mild cool false true play 0.0 bits * 3/5 0.0 bits * 2/5 0.92 bits * 3/5 0.0 bits * 2/5 1.0 bits * 2/5 0.0 bits * 1/5 1.0 bits * 2/5 don’t play + = 0.0 bits + = 0.40 bits + = 0.95 bits gain: 0.97 bits gain: 0.57 bits gain: 0.02 bits ID3 outlook sunny overcast rainy 0.97 bits maximal information gain

28. humidity temperature windy high normal hot mild cool false true  play 1.0 bits *2/5 0.92 bits * 3/5 0.0 bits * 3/5 0.92 bits * 3/5 1.0 bits * 2/5 0.0 bits * 2/5 don’t play + = 0.95 bits + = 0.95 bits + = 0.0 bits gain: 0.02 bits gain: 0.02 bits gain: 0.97 bits ID3 outlook overcast sunny rainy 0.97 bits humidity high normal

29. ID3 play don’t play outlook sunny overcast rainy Yes humidity windy high true false normal Yes No No Yes

30. Outlook Sunny Overcast Rain Humidity Yes Wind High Normal Strong Weak No Yes No Yes Converting Tree to Rules R1: IF (Outlook=Sunny) AND (Humidity=High) THEN Play=No R2: IF (Outlook=Sunny) AND (Humidity=Normal) THEN Play=Yes R3: IF (Outlook=Overcast) THEN Play=Yes R4: IF (Outlook=Rain) AND (Wind=Strong) THEN Play=No R5: IF (Outlook=Rain) AND (Wind=Weak) THEN Play=Yes

31. Tugas : Learning decision trees Problem: decide whether to wait for a table at a restaurant, based on the following attributes: • Alternate: is there an alternative restaurant nearby? • Bar: is there a comfortable bar area to wait in? • Fri/Sat: is today Friday or Saturday? • Hungry: are we hungry? • Patrons: number of people in the restaurant (None, Some, Full) • Price: price range ($, $$, $$$) • Raining: is it raining outside? • Reservation: have we made a reservation? • Type: kind of restaurant (French, Italian, Thai, Burger) • WaitEstimate: estimated waiting time (0-10, 10-30, 30-60, >60)

32. Attribute-based representations • Examples described by attribute values (Boolean, discrete, continuous) • E.g., situations where I will/won't wait for a table: • Classification of examples is positive (T) or negative (F)

33. Tugas Decision Tree