1 / 29

Bayesian Belief Networks Chapter 2 (Duda et al.) – Section 2.11

Bayesian Belief Networks Chapter 2 (Duda et al.) – Section 2.11. CS479/679 Pattern Recognition Dr. George Bebis. Statistical Dependences Between Variables. Many times, the only knowledge we have about a distribution is which variables are or are not dependent .

luna
Télécharger la présentation

Bayesian Belief Networks Chapter 2 (Duda et al.) – Section 2.11

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Belief NetworksChapter 2 (Duda et al.) – Section 2.11 CS479/679 Pattern RecognitionDr. George Bebis

  2. Statistical Dependences Between Variables • Many times, the only knowledge we have about a distribution is which variables areorare not dependent. • Such dependencies can be represented efficiently using a Bayesian Belief Network (or Belief Net or Bayesian Net). • Bayesian Nets allow us to represent a joint probability density p(x,y,z,…) efficiently using dependency relationships.

  3. Example of Dependencies • State of an automobile • Engine temperature • Brake fluid pressure • Tire air pressure • Wire voltages • Causally related variables • Coolant temperature • Engine temperature • NOT causally related variables • Engine oil pressure • Tire air pressure

  4. Applications • Microsoft (Answer Wizard, Print Troubleshooter) • US Army: SAIP (Battalion Detection from SAR, IR etc.) • NASA: Vista (DSS for Space Shuttle) • GE: Gems (real-time monitoring of utility generators)

  5. Definitions and Notation • A belief net is usually a Directed Acyclic Graph (DAG) • Each node represents one of the system variables. • Each variable can assume certain states (i.e., values).

  6. Relationships Between Nodes • A link joining two nodes is directional and represents a causal influence (e.g., X depends on A or A influences X) • Influences could be direct or indirect (e.g., A influences X directly and A influences C indirectly through X).

  7. Conditional Probability Tables • Each variable is associated with a set of weights which represent prior or conditional probabilities (discrete or continuous) . Probabilities sum to 1

  8. Parent/Children Nodes • Parent nodes P of X • the nodes before X (connected to X) • Children nodes C of X: • the nodes after X (X is connected to them) Markov property: each node is conditionally independent of its ancestors given its parents.

  9. Computing Joint Probabilities • Using the chain rule, the joint probability of a set of variables x1, x2, …, xn is give as: • The conditional independence relationships encoded in the Bayesian network state that a node xi is conditionally independent of its ancestors given its parents πi : = much simpler!

  10. Computing Joint Probabilities (cont’d) • We can compute the probability of any configuration of variables in the joint density distribution, e.g.: P(a3, b1, x2, c3, d2)=P(a3)P(b1)P(x2 /a3,b1)P(c3 /x2)P(d2 /x2)= 0.25 x 0.6 x 0.4 x 0.5 x 0.4 = 0.012

  11. Computing the Probability at a Node • e.g., determine the probability at D

  12. Computing the Probability at a Node (cont’d) • e.g., determine the probability at H: =

  13. Two Fundamental Problems in Bayesian Nets • Evaluation (inference) problem: Given the model and the values of the observed variables (evidence), estimate the values of the hidden nodes. • Learning problem: Given training data and prior information (e.g., expert knowledge, causal relationships), estimate the network structure, or the parameters of the probability distributions, or both.

  14. Evaluation (Inference) Problem • In general, if X denotes the query variables and e denotes the evidence, then where α=1/P(e) is a constant of proportionality.

  15. Evaluation (Inference) Problem (cont’d) • Prediction, or top-down reasoning: observing the “roots” and trying to predict the effects. • Diagnosis, or bottom-up reasoning: observing the “leaves” and trying to infer the values of the hidden causes. • Exact inference is an NP-hard problem because the number of terms in the summations (integrals) for discrete (continuous) variables grows exponentially with increasing number of variables.

  16. Evaluation (Inference) Problem (cont’d) • Some restricted classes of networks (e.g., singly connected networks where there is no more than one path between any two nodes) can be efficiently solved in time linear in the number of nodes. • However, approximate inference methods such as • sampling (Monte Carlo) methods • variational methods • loopy belief propagation have to be used for most of the cases.

  17. Example • Classify a fish given that the fish is light (c1) and was caught in south Atlantic (b2) -- no evidence about what time of the year the fish was caught nor its thickness.

  18. Example (cont’d)

  19. Example (cont’d)

  20. Example (cont’d) • Similarly, P(x2 / c1,b2)=α 0.066 • Normalize probabilities (not needed necessarily): P(x1 /c1,b2)+ P(x2 /c1,b2)=1 (α=1/0.18) P(x1 /c1,b2)= 0.73 P(x2 /c1,b2)= 0.27 salmon

  21. Another Example • You have a new burglar alarm installed at home. • It is fairly reliable at detecting burglary, but also sometimes responds to minor earthquakes. • You have two neighbors, Ali and Veli, who promised to call you at work when they hear the alarm.

  22. Another Example (cont’d) • Ali always calls when he hears the alarm, but sometimes confuses telephone ringing with the alarm and calls too. • Veli likes loud music and sometimes misses the alarm. • Given the evidence of who has or has not called, we would like to estimate the probability of a burglary.

  23. Another Example (cont’d) • Design the structure of the Bayesian net. • What are the main variables? • Alarm, Burglary, Earthquake, Ali calls, Veli calls • What are the conditional dependencies among them? • Burglary (B) and earthquake (E) directly affect the probability of the alarm (A) going off • Whether or not Ali calls (AC) or Veli calls (VC) depends only on the alarm.

  24. Another Example (cont’d)

  25. Another Example (cont’d) • What is the probability that the alarm has sounded but neither a burglary nor an earthquake has occurred, and both Ali and Veli call?

  26. Another Example (cont’d) • What is the probability that there is a burglary given that Ali calls? • What about if Veli also calls right after Ali hangs up?

  27. Medical Diagnosis Example Uppermost nodes:biological agents (bacteria, virus) Intermediate nodes:diseases Lowermost nodes:symptoms • Given some evidence (biological agents, symptoms), find most likely disease.

  28. Learning Problem • The simplest situation is the one where the network structure is completely known (either specified by an expert or designed using causal relationships between the variables). • Other situations with increasing complexity are: • known structure but unobserved variables • unknown structure with observed variables, and • unknown structure with unobserved variables. Four cases in Bayesian network learning.

  29. Naïve Bayesian Network • When dependency relationships among features are unknown, we can assume that features are conditionally independent given the class: • Simple assumption but usually works well in practice! Naïve Bayesian Network:

More Related