Sensitivity Analysis in Bayesian Networks

1. Sensitivity Analysis inBayesian Networks Adnan Darwiche Computer Science Department http://www.cs.ucla.edu/~darwiche A. Darwiche

2. Bayesian network classifiers • Given: • A Bayesian network N • A class variable C • A set of variables, attributes E = {E1, …, En} • Each instantiation e is called an instance • A probability threshold p • Define Bayesian network classifier F: A. Darwiche

3. Naïve Bayes Classifiers Pregnant? (P) Urine test Blood test Scanning test (U) (B) (S) Which sets of test results confirm pregnancy, with probability no less than 90%? A. Darwiche

4. Reasoning about Bayesian network classifiers • Given N and N’, do they induce the same classifier? • If not, which, and how many, instances do they disagree on? • Given N, what are the allowable changes to a CPT which will not change the current classifier? • How many distinct classifiers can be induced by changing some CPT? A. Darwiche

5. Reasoning about Bayesian network classifiers • We can answer these questions by enumerating all instances e explicitly • However, this is often infeasible given the exponential number of instances • Instead, we propose to build a tractable logical representation of the classifier FN • This allows us to answer the above questions in time linear in the size of the representation A. Darwiche

6. P Pr(p) Pregnant? yes 0.87 (P) no 0.13 Urine test Blood test Scanning test (U) (B) (S) P U Pr(u|p) P B Pr(b|p) P S Pr(s|p) yes -ve 0.27 yes -ve 0.36 yes -ve 0.10 no +ve 0.107 no +ve 0.106 no +ve 0.01 From Numbers to Decisions Test results: U, B, S Decision Function + Probabilistic Inference Yes, No A. Darwiche

7. P Pr(p) Pregnant? yes 0.87 (P) no 0.13 Urine test Blood test Scanning test (U) (B) (S) P U Pr(u|p) P B Pr(b|p) P S Pr(s|p) yes -ve 0.27 yes -ve 0.36 yes -ve 0.10 no +ve 0.107 no +ve 0.106 no +ve 0.01 From Numbers to Decisions Situation: U=+ve, B=-ve, S=-ve U +ve - ve B - ve +ve S +ve + Probabilistic Inference - ve Yes No Ordered Decision Diagram A. Darwiche

8. X1 X2 X2 X3 X3 1 0 Binary Decision Diagram Test-once property A. Darwiche

9. Improving Reliability of Sensors Yes if > 90% Pregnant? (P) Urine test Blood test Scanning test (U) (B) (S) Currently False negative 27.0% False positive 10.7% Same decisions (in all situations) if new test is: False negative 10% False positive 5% Different decisions (in some situations) if new test: False negative 5% False positive 2.5% Can characterize these situations, compute their likelihood, analyze their properties A. Darwiche

10. Adding New Sensors Yes if > 90% Pregnant? (P) Urine test Blood test Scanning test New test (U) (B) (S) (N) Same decisions (in all situations) if: False negative 40% False positive 20% Different (in some situations) decisions if: False negative 20% False positive 10% Can characterize these situations, compute their likelihood, analyze their properties A. Darwiche

11. Naïve Bayes classifier Class variable C Attributes E A. Darwiche

12. Naïve Bayes classifier Prior log-odds Weight of evidence ei A. Darwiche

13. Changing the prior log-odds in a naïve Bayes classifier • If we change the CPT of C, thereby changing the prior log-odds from log O(c) to log O’(c), will we still have the same classifier? A. Darwiche

14.   Changing the prior log-odds in a naïve Bayes classifier A. Darwiche

15. Equivalence of NB classifiers A. Darwiche

16. Equivalence of NB classifiers Change prior of P • Equivalent iff prior of P in FN’[0.684, 0.970) A. Darwiche

17. Path 1 Path 2 Sub-ODD D1 Sub-ODD D2 A. Darwiche

18. Path 1 Path 2 Sub-ODD D1 = D2 A. Darwiche

19. Theoretical results of algorithm • Space complexity: • Total number of nodes in the ODD  O(bn/2) • Time complexity: • O(nbn/2) • Improves greatly over brute-force approach: • Total number of instances = O(bn) A. Darwiche

20. Experimental results of algorithm A. Darwiche

21. http://reasoning.cs.ucla.edu/samiam/ A. Darwiche

22. A. Darwiche