Probabilistic Reasoning; Network-based reasoning

Probabilistic Reasoning;Network-based reasoning Set 7 ICS 179, Spring 2010

= A = B = A = C Propositional Reasoning Example: party problem • If Alex goes, then Becky goes: • If Chris goes, then Alex goes: • Question: Is it possible that Chris goes to the party but Becky does not? Chavurah 5/8/2010

P(A|W=bad)=.9 W A P(C|W=bad)=.1 W C W B P(B|W=bad)=.5 P(W) W A B C P(A|W) P(B|W) P(C|W) Probabilistic Reasoning Party example: the weather effect • Alex is-likely-to-go in bad weather • Chris rarely-goes in bad weather • Becky is indifferent but unpredictable Questions: • Given bad weather, which group of individuals is most likely to show up at the party? • What is the probability that Chris goes to the party but Becky does not? P(W,A,C,B) = P(B|W) · P(C|W) · P(A|W) · P(W) P(A,C,B|W=bad) = 0.9 · 0.1 · 0.5 Chavurah 5/8/2010

P(W) P(W) B B A A C C W W A→B A→B C→A C→A P(B|W) P(B|W) P(C|W) P(C|W) P(A|W) P(A|W) B B A A C C Mixed Probabilistic and Deterministic networks PN CN Query: Is it likely that Chris goes to the party if Becky does not but the weather is bad? Semantics? Algorithms? Chavurah 5/8/2010

The problem True propositions Uncertain propositions Q: Does T fly? P(Q)? Logic?....but how we handle exceptions Probability: astronomical

Alpha and beta are events

Burglary is independent of Earthquake

Earthquake is independent of burglary

P(S) P(C|S) P(B|S) • C B D=0 D=1 • 0 0 0.1 0.9 • 0 1 0.7 0.3 • 1 0 0.8 0.2 • 1 1 0.9 0.1 CPD: P(X|C,S) P(D|C,B) Conditional Independencies Efficient Representation Bayesian Networks: Representation Smoking lung Cancer Bronchitis X-ray Dyspnoea P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)

Bayesian networks Chapter 14 , Russel and Norvig Section 1 – 2

Outline • Syntax • Semantics

Example • Topology of network encodes conditional independence assertions: • Weather is independent of the other variables • Toothache and Catch are conditionally independent given Cavity

Example • I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar? • Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls • Network topology reflects "causal" knowledge: • A burglar can set the alarm off • An earthquake can set the alarm off • The alarm can cause Mary to call • The alarm can cause John to call

Example contd.

Compactness • A CPT for Boolean Xi with k Boolean parents has 2k rows for the combinations of parent values • Each row requires one number p for Xi = true(the number for Xi = false is just 1-p) • If each variable has no more than k parents, the complete network requires O(n · 2k) numbers • I.e., grows linearly with n, vs. O(2n) for the full joint distribution • For burglary net, 1 + 1 + 4 + 2 + 2 = 10 numbers (vs. 25-1 = 31)

Semantics The full joint distribution is defined as the product of the local conditional distributions: P (X1, … ,Xn) = πi = 1P (Xi | Parents(Xi)) e.g., P(j  m  a b e) = P (j | a) P (m | a) P (a | b, e) P (b) P (e) n

Constructing Bayesian networks • 1. Choose an ordering of variables X1, … ,Xn • 2. For i = 1 to n • add Xi to the network • select parents from X1, … ,Xi-1 such that P (Xi | Parents(Xi)) = P (Xi | X1, ... Xi-1) This choice of parents guarantees: P (X1, … ,Xn) = πi =1P (Xi | X1, … , Xi-1) (chain rule) = πi =1P (Xi | Parents(Xi)) (by construction)

Example • Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)?

Example • Suppose we choose the ordering M, J, A, B, E P(J | M) = P(J)? No P(A | J, M) = P(A | J)?P(A | J, M) = P(A)?

Probabilistic Reasoning; Network-based reasoning