Créer une présentation
Télécharger la présentation

Télécharger la présentation
## Independence

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Independence**• X, Y are independent if changing X does not affect the probability of Y, and vice versa. • Pr(X, Y) = Pr(X)∙Pr(Y) • Pr(X | Y) = Pr(X) • Pr(Y | X) = Pr(Y)**Independent or Dependent?**• Doritos for dinner, getting the flu • Miles per gallon, driving habits • Alice and Bob tossing two separate coins • Alice and Bob tossing the same coin**Independent or Dependent?**• Doritos for dinner, getting the flu • Independent • Miles per gallon, driving habits • Dependent • Alice and Bob tossing two separate coins • Independent • Alice and Bob tossing the same coin • Dependent?**Alice and Bob toss the same coin**• Coin may be biased • If Alice’s toss is Heads, more likely that Bob’s toss will be Heads.**Alice and Bob toss the same coin**• Coin may be biased • If Alice’s toss is Heads, more likely that Bob’s toss will be Heads. • The coin tosses are dependent on what separate event?**Alice and Bob toss the same coin**• Coin may be biased • If Alice’s toss is Heads, more likely that Bob’s toss will be Heads. • The coin tosses are dependent on a separate event: Coin is biased. • Given that the coin is biased, Alice’s toss is independent from Bob’s toss.**Alice and Bob toss the same coin**• A = outcome of Alice’s toss • B = outcome of Bob’s toss • C = event that the coin is biased towards Heads**Alice and Bob toss the same coin**• A = outcome of Alice’s toss • B = outcome of Bob’s toss • C = event that the coin is biased towards Heads • A and B areconditionally independent given C if: • Pr(A| C Λ B)=Pr(A | C) • Pr(B| C ΛA)=Pr(B | C)**Bayes Networks**• Somewhat simplified • Few assumptions • A bit mathy**Joint Probability Distributions**For n variables, how many probabilities?**Joint Probability Distributions**For n variables, how many probabilities? 2n**Joint Probability Distributions**For n variables, how many probabilities? 2n Joint Probability Distributions require a lot of storage.**Bayesian Network**• Data Structure that represents dependencies among random variables of a joint probability distribution.**Bayesian Network**• Data Structure that represents dependencies among random variables of a joint probability distribution. • Directed Acyclic Graph with nodes and edges: • Node for every variable X**Bayesian Network**• Data Structure that represents dependencies among random variables of a joint probability distribution. • Directed Acyclic Graph with nodes and edges: • Node for every variable X • Edge (Y, X) from Y to X if X depends on Y “Y is a parent of X” Y X**Bayesian Network for Flu example (on Board)**Bayes Network contains 5 probabilities instead of 8**Find Probability of an Event (B)**Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, B**Find Probability of an Event (B)**Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y B**Find Probability of an Event (B)**Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y • Notation: Pr(B) = P(B, Y=) + P(B, Y=) + …+ P(B, Y=) B**Find Probability of an Event (B)**Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y • Notation: Pr(B) = P(B, Y=) + P(B, Y=) + …+ P(B, Y=) = Pr(B | Y=)Pr(Y=)+ Pr(B | Y=)Pr(Y=)+ …+ Pr(B | Y=)Pr(Y=) B**Find Probability of an Event (B)**Y • Simplest Case (Case 1): • B has one parent, Y • Y has no parents • Y has possible values , , …, • Take conditional probability of B over all possible values of Y • Notation: Pr(B) = P(B, Y=) + P(B, Y=) + …+ P(B, Y=) = Pr(B | Y=)Pr(Y=)+ Pr(B | Y=)Pr(Y=)+ …+ Pr(B | Y=)Pr(Y=) = B**Example-Flu Network**Probability of Sore Throat?**Example-Flu Network**Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu.**Example-Flu Network**Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu: Pr(S) = P(S | F=)Pr(F=) + P(S | F=) Pr(F=) + … +P(S | F=) Pr(F=)**Example-Flu Network**Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu: Pr(S) = P(S | F=)Pr(F=) + P(S | F=) Pr(F=) + … +P(S | F=) Pr(F=) Possible values of F (Flu)?**Example-Flu Network**Probability of Sore Throat? Flu is parent of Sore Throat. Compute Probability of Sore Throat over all possible values of Flu: Pr(S) = P(S | F=)Pr(F=) + P(S | F=) Pr(F=) + … +P(S | F=) Pr(F=) Possible values of F (Flu)? Just True/False Pr(S) = Pr(S | F)Pr(F) + Pr(S | ⌐ F)Pr(⌐ F) = (.12)(.2) + (.08)(.8)**Case 2: B has multiple parents**Y X Z What if B has multiple parents? B**Case 2: B has multiple parents**Y X Z What if B has multiple parents? X, Y, Z are parents of B with possible values , , …, , , …, , , …, Assume X, Y, Z are independent of each other B**Case 2: B has multiple parents**Y X Z What if B has multiple parents? X, Y, Z are parents of B with possible values , , …, , , …, , , …, Assume X, Y, Z are independent of each other Consider all possible combinations of X, Y, Z. B**Probability of an Event (B)**Pr(B) = P(B, X=Y=) + P(B, X=Y=) + P(B, X=Y=) + …+ P(B, X=Y=)**Probability of an Event (B)**Pr(B) = P(B, X=Y=) + P(B, X=Y=) + P(B, X=Y=) + …+ P(B, X=Y=) = Pr(B|X=Y=)Pr(X=Y=)+ Pr(B|X=Y=)Pr(X=Y=)+ Pr(B|X=Y=)Pr(X=Y=)+…+ Pr(B|X=Y=)Pr(X=Y=)**More Complex Example**• You install new burglar alarm • Reliable at detecting burglary but also rings occasionally during a minor earthquake • Two neighbors Jack and Mary who promise to call you if they hear the alarm • Jack almost always calls when he hears alarm but occasionally confuses phone with alarm and sometimes calls when he hears the phone • Mary is hard of hearing: occasionally does not hear the alarm**But First…How does this relate to AI?**Want to create intelligent agents that can make rational decisions given uncertainty. Jack and Mary clearly lack uncertainty.**More Complex Example**• You install new burglar alarm • Reliable at detecting burglary but also rings occasionally during a minor earthquake • Two neighbors Jack and Mary who promise to call you if they hear the alarm • Jack almost always calls when he hears alarm but occasionally confuses phone with alarm and sometimes calls when he hears the phone • Mary is hard of hearing: occasionally does not hear the alarm**Burglary Alarm Example**• (Bayes Network on the board) • Probability that alarm goes off?**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q?**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q? 4**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q)**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)(.001)(.002)**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)(.001)(.002) + (.94)(.001)(.998) +**Pr(A) = P(A, B, Q)**How many possible combinations of B, Q? 4 = Pr(A|B Λ Q)Pr(B Λ Q) + Pr(A|B Λ ⌐Q)Pr(B Λ ⌐Q) + Pr(A| ⌐ B Λ Q)Pr(⌐ B Λ Q) + Pr(A| ⌐ B Λ ⌐ Q)Pr(⌐ B Λ ⌐ Q) =(.95)(.001)(.002) + (.94)(.001)(.998) + (.29)(.999)(.002) + (.001)(.999)(.998)**How to find probability of 2 events A, B?**Assume A and B are independent.**How to find probability of 2 events A, B?**Assume A and B are independent. Suppose X, Y, Z are parents of A with possible values: X: Y: Z: P, Q, R are parents of B with possible values: P: Q: R: Y Q X Z P R A B**How to find probability of 2 events A, B?**Assume A and B are independent. Suppose X, Y, Z are parents of A with possible values: X: Y: Z: P, Q, R are parents of B with possible values: P: Q: R: Consider all possible combinations of X, Y, Z, P, Q, R Y Q X Z P R A B**X, Y, Z, P, Q, R can each take on 3 values.**How many possible combinations?**X, Y, Z, P, Q, R can each take on 3 values.**How many possible combinations? 36 E1 E2 … E36 Pr(A, B) = P(A, B, E1) + P(A, B, E1) + … P(A, B, E36)**Case 3: “Chain” of Parents**Y What if B has a parent X, and X has a parent Y? X B