Probabilistic Reasoning With Bayes’ Rule

Probabilistic Reasoning With Bayes’ Rule Outline: Motivation Generalizing Modus Ponens Bayes’ Rule Applying Bayes’ Rule Odds Odds-Likelihood Formulation of Bayes’ Rule Combining independent items of evidence General combination of evidence Benefits of Bayes nets for expert systems CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Motivation Logical reasoning has limitations: It requires that assumptions be considered “certain”. It typically uses general rules. General rules that are reliable may be difficult to come by. Logical reasoning can be awkward for certain structured domains such as time and space. CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Generalizing Modus Ponens Modus Ponens: P -> Q P ---------- Q Bayes’ Rule: (general idea) If P then sometimes Q P ------------------------------- Maybe Q (Bayes’ rule lets us calculate the probability of Q, taking P into account.) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Bayes’ Rule E: Some evidence exists, i.e., a particular condition is true H: some hypothesis is true. P(E|H) = probability of E given H. P(E|~H) = probability of E given not H. P(H) = probability of H, independent of E. P(E|H) P(H) P(H|E) = ----------------- P(E) P(E) = P(E|H) P(H) + P(E|~H)(1 - P(H)) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Applying Bayes’ Rule E: The patient’s white blood cell count exceeds 110% of average. H: The patient is infected with tetanus. P(E|H) = 0.8 class-conditional probability P(E|~H) = 0.3 “ P(H) = 0.01 prior probability posterior probability: P(E|H) P(H) (0.8) (0.01) 0.008 P(H|E) = ----------------- =-------------------------------- = ------- = 0.0262 P(E) (0.8) (0.01) + (0.3)(0.99) 0.305 P(E) = P(E|H) P(H) + P(E|~H)(1 - P(H)) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Odds Odds are 10 to 1 it will rain tomorrow. 10 10 P(rain) = ---------- = ------- 10 + 1 11 Suppose P(A) = 1/4 Then O(A) = (1/4) / (3/4) = 1/3 P(A) P(A) in general: O(A) = -------- = ----------- P(~A) 1 - P(A) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Odds-LikelihoodForm of Bayes’ Rule E: The patient’s white blood cell count exceeds 110% of average. H: The patient is infected with tetanus. O(H) = 0.01/0.99 O(H|E) = λ O(H) lambda is called the sufficiency factor. O(H|~E) = λ’ O(H) lambda prime is called the necessity factor. CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

The Monty Hall Problem From the Wikipedia CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

The Monty Hall Problem There are three doors: a red door, green door, and blue door. Behind one is a car, and behind the other two are goats. You get to keep whatever is behind the door you choose. You choose a door (say, red). The host opens one of the other doors (say, green), which reveals a goat. The host says, “Would you like to select the OTHER door?” Should you switch? CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Discussion A: car is behind red door B: car is behind green door C: car is behind blue door P(A) = P(B) = P(C) = 1/3 Suppose D: you choose the red door, and the host opens the green door revealing a goat. Is P(C|D) = ½ ???? (No) Why not? What is P(C|D)? CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Bayes Nets A practical way to manage probabilistic inference when multiple variables (perhaps many) are involved. CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Why Bayes Networks? Reasoning about events involving many parts or contingencies generally requires that a joint probability distribution be known. Such a distribution might require thousands of parameters. Modeling at this level of detail is typically not practical. Bayes Nets require making assumptions about the relevance of some conditions to others. Once the assumptions are made, the joint distribution can be “factored” so that there are many fewer separate parameters that must be specified. CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Review of Bayes’ Rule E: Some evidence exists, i.e., a particular condition is true H: some hypothesis is true. P(E|H) = probability of E given H. P(E|~H) = probability of E given not H. P(H) = probability of H, independent of E. P(E|H) P(H) P(H|E) = ----------------- P(E) P(E) = P(E|H) P(H) + P(E|~H)(1 - P(H)) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Combining Independent Items of Evidence E1: The patient’s white blood cell count exceeds 110% of average. E2: The patient’s body temperature is above 101oF. H: The patient is infected with tetanus. O(H) = 0.01/0.99 O(H|E1) = λ1 O(H) sufficiency factor for high white cell count. O(H|E2) = λ2 O(H) sufficiency factor for high body temp. Assuming E1 and E2 are independent: O(H|E1 E2) = λ1 λ2O(H) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Bayes Net Example P(A) = 0.2 A P(C|A) = 0.3 P(C|~A) = 0.1 P(B|A) = 0.5 P(B|~A) = 0.15 B C A: Accident (An accident blocked traffic on the highway.) B: Barb Late (Barbara is late for work). C: Chris Late (Christopher is late for work). CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Forward Propagation(from causes to effects) P(A) = 0.2 A P(C|A) = 0.3 P(C|~A) = 0.1 P(B|A) = 0.5 P(B|~A) = 0.15 B C Suppose A (there is an accident): Then P(B|A) = 0.5 P(C|A) = 0.3 Suppose ~A (no accident): Then P(B|~A) = 0.15 P(C|A) = 0.1 (These come directly from the given information.) CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Marginal Probabilities(using forward propagation) P(A) = 0.2 A P(C|A) = 0.3 P(C|~A) = 0.1 P(B|A) = 0.5 P(B|~A) = 0.15 B C Then P(B) = probability Barb is late in any situation = P(B|A) P(A) + P(B|~A) P(~A) = (0.5)(0.2) + (0.15)(0.8) = 0.22 Similarly P(C) = probability Chris is late in any situation = P(C|A) P(A) + P(C|~A) P(~A) = (0.3)(0.2) + (0.1)(0.8) = 0.14 Marginalizing means eliminating a contingency by summing the probabilities for its different cases (here A and ~A). CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Backward Propagation: “diagnosis”(from effects to causes) P(A) = 0.2 A P(C|A) = 0.3 P(C|~A) = 0.1 P(B|A) = 0.5 P(B|~A) = 0.15 B C Suppose B (Barb is late) What’s the probability of an accident on the highway? Use Bayes’ rule: Then P(A|B) = P(B|A) P(A) / P(B) = 0.5 * 0.2 / (0.5 * 0.2 + 0.15 * 0.8) = 0.1 / 0.22 = 0.4545 CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Revising Probabilities of Consequences P(A|B) = 0.4545 A P(C|A) = 0.3 P(C|~A) = 0.1 P(C|B) = ??? P(B|A) = 0.5 P(B|~A) = 0.15 B C Suppose B (Barb is late). What’s the probability that Chris is also late, given this information? We already figured that P(A|B) = 0.4545 P(C|B) = P(C|A) P(A|B) + P(C|~A) P(~A|B) = (0.3)(0.4545) + (0.1)(0.5455) = 0.191 somewhat higher than P(C)=0.14 CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Benefits of Bayes Nets • The joint probability distribution normally requires 2n – 1 independent parameters. • With Bayes Nets we only specify these parameters: • “root” node probabilities. • e. g., P(A=true) = 0.2; P(A=false)=0.8. • For each non-root node, a table of 2k values, where k is the number of parents of that node. • Typically k < 5. • 3. Propagating probabilities happens along the paths in the net. With a full joint prob. dist., many more computations may be needed. CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

Probabilistic Reasoning With Bayes’ Rule