1 / 58

10601 Machine Learning

September 2, 2009. 10601 Machine Learning. Recitation 2 Öznur Taştan. Logistics. Homework 2 is going to be out tomorrow. It is due on Sep 16, Wed. There is no class on Monday Sep 7 th (Labor day) Those who have not return Homework 1 yet

Télécharger la présentation

10601 Machine Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. September 2, 2009 10601 Machine Learning Recitation 2 Öznur Taştan

  2. Logistics • Homework 2 is going to be out tomorrow. It is due on Sep 16, Wed. • There is no class on Monday Sep 7th (Labor day) • Those who have not return Homework 1 yet • For details of how to submit the homework policy please check : http://www.cs.cmu.edu/~ggordon/10601/hws.html

  3. Outline • We will review • Some probability and statistics • Some graphical models • We will not go over Homework 1 • Since the grace period has not ended yet. • Solutions will be up next week on the web page.

  4. We’ll play a game: Catch the goof! • I’ll be the sloppy TA… will make ‘intentional’ mistakes • You’ll catch those mistakes and correct me! Slides with mistakes are marked with Correct slides are marked with

  5. Catch the goof!!

  6. Law of total probability Given two discrete random variables X and Y X takes values in Y takes values in

  7. Law of total probability Given two discrete random variables X and Y X takes values in Y takes values in

  8. Law of total probability Given two discrete random variables X and Y X takes values in Y takes values in

  9. Law of total probability Given two discrete random variables X and Y X takes values in Y takes values in

  10. Law of total probability Given two discrete random variables X and Y Joint probability Marginal probability Conditional probability of X conditioned on Y

  11. Law of total probability Given two discrete random variables X and Y Formulas are fine. Anything wrong with the names? Joint probability Marginal probability Conditional probability of X conditioned on Y

  12. Law of total probability Given two discrete random variables X and Y Joint probability of X,Y Marginal probability Conditional probability of X conditioned on Y Marginal probability

  13. In a strange world Two discrete random variables X and Y take binary values Joint probabilities

  14. In a strange world Two discrete random variables X and Y take binary values Joint probabilities Should sum up to 1

  15. The world seems fine Two discrete random variables X and Y take binary values Joint probabilities

  16. What about the marginals? Joint probabilities Marginal probabilities

  17. This is a strange world Joint probabilities Marginal probabilities

  18. In a strange world Joint probabilities Marginal probabilities

  19. This is a strange world Joint probabilities Marginal probabilities

  20. Let’s have a simple problem Joint probabilities Marginal probabilities

  21. Conditional probabilities What is the complementary event of P(X=0|Y=1) ? P(X=1|Y=1) OR P(X=0|Y=0)

  22. Conditional probabilities What is the complementary event of P(X=0|Y=1) ? P(X=1|Y=1) OR P(X=0|Y=0)

  23. The game ends here.

  24. Independent number of parameters Assume X and Y take Boolean values {0,1}: • How many independent parameters do you need to fully specify: • marginal probability of X? • the joint probability of P(X,Y)? • the conditional probability of P(X|Y)?

  25. Independent number of parameters Assume X and Y take Boolean values {0,1}: • How many independent parameters do you need to fully specify: • marginal probability of X? P(X=0) 1 parameter only [ because P(X=1)+P(X=0)=1 ] • the joint probability of P(X,Y)? P(X=0, Y=0) 3 parameters P(X=0, Y=1) P(X=1, Y=0) • the conditional probability of P(X|Y)?

  26. Number of parameters • Assume X and Y take Boolean values {0,1}? • How many independent parameters do you need to fully specify marginal probability of X? P(X=0) 1 parameter only P(X=1)= 1-P(X=0) • How many independent parameters do you need to fully specify the joint probability of P(X,Y)? P(X=0, Y=0) 3 parameters P(X=0, Y=1) P(X=1, Y=0) • How many independent parameters do you need to fully specify the conditional probability of P(X|Y)? P(X=0|Y=0) 2 parameters P(X=0|Y=1)

  27. Number of parameters • What about P(X | Y,Z) , how many independent parameters do you need to be able to fully specify the probabilities? Assume each RV takes: m values P(X | Y,Z) q values n values

  28. Number of parameters • What about P(X | Y,Z) , how many independent parameters do you need to be able to fully specify the probabilities? Assume each RV takes: m values P(X | Y,Z) q values n values Number of independent parameters: (m-1)*nq

  29. Graphical models • A graphical model is a way of representing probabilistic relationships between random variables • Variablesare represented by nodes: • Edges indicates probabilistic relationships: You miss the bus Arrive class late

  30. Serial connection Is X and Z independent? ?

  31. Serial connection Is X and Z independent? X and Z are not independent

  32. Serial connection Is X conditionally independent of Z given Y? ?

  33. Serial connection Is X conditionally independent of Z given Y? Yes they are independent

  34. How can we show it? Is X conditionally independent of Z given Y?

  35. An example case Studied late last night Wake up late Arrive class late

  36. Common cause Age Shoe Size Gray Hair X and Y are not marginally independent X and Y are conditionally independent given Z

  37. Explaining away Flu Allergy Z X Y Sneeze X and Z marginally independent X and Z conditionally dependent given Y

  38. D-separation • X and Z are conditionally independent given Y if Y d-separates X and Z Neither Y nor its descendants should be observed Path between X and Z is blocked by Y

  39. D-separation example Is B, C independent given A?

  40. D-separation example Is B, C independent given A? Yes

  41. D-separation example Observed, A blocks the path Is B, C independent given A? Yes

  42. Observed, A blocks the path Is B, C independent given A? Yes not observed neither its descendants

  43. D-separation example Is A, F independent given E?

  44. Is A, F independent given E? Yes

  45. Is A, F independent given E? Yes

  46. Is C, D independent given F?

  47. Is C, D independent given F? No

  48. Is A, G independent given B and F?

  49. Is A, G independent given B and F? Yes

  50. Naïve Bayes Model J D C R J: The person is a junior D: The person knows calculus C: The person leaves in campus R: Saw the “Return of the King” more than once

More Related