Create Presentation
Download Presentation

Download Presentation
## Robotic Rational Reasoning! Lecture 2

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Robotic Rational Reasoning! Lecture 2**P.H.S. Torr**Summary of last lecture**• We talked about the problem of induction and how it might relate to AI. • We saw probability might be the basis of reasoning about the world. • We saw there were problems with the attempts to make objective interpretations of probability. • Venn • Von Mises (frequentists) • Popper (propensity)**This lecture**• We will develop a system of probability that can be used within AI. • We shall show how this can be used to reason about events and decide actions. • How to represent our belief in propositions by numbers • How to make action on the basis of those beliefs • Once we have done this we want to show how to act rationally, so as to optimize utility.**Belief**• Why is belief important? • Consider again the example of the coin hidden in one of my hands. • Your belief in which hand it is in, is different from mine. • Can we make this more formal so that we can use belief in practise?**We talked about ‘stronger’ and ‘weaker’ belief; and**being ‘more or less confident’. Can we make sense of this talk? That is, can we quantify partial belief, and give a numerical basis to these comparative judgements? Assign numbers to measure the strength of a belief; Prove that those numbers must be, formally, probabilities; and Provide a rule for updating beliefs given new evidence. These three achievements constitute Bayesian epistemology. Quantifying Belief**Task 1: Measuring Belief**• We turn first to the task of assigning numbers to measure belief. • Is it possible to measure belief in some rational way?**Theories of Belief**• Logical • Keynes, Carnap, Cox, Jaynes • Preference over actions • Ramsey, Savage • Betting • De Finetti**Logical theory of belief**• Here logic is extended from binary valued to include uncertainty over propositions. • This theory was first put forward by economist Keynes and then given rigour by R.T. Cox. • It is convincing but difficult maths.**Subjective theory of belief based on Actions**• By Ramsey and Savage. • From a person expressing preferences between actions one can derive the whole of probability theory, a remarkable feat. • Again highly mathematical.**Gambles and Degrees of Belief**• Probability has a long association with games of chance. • One way that we have talked about probabilities was in terms of rolls of die and tosses of coins. • A better way to represent degrees of belief will be in terms of gambles. • A gamble is simply a choice between two possibilities or options. • The idea is to: • Assign cash payoffs to options, and then • Assign numbers to which of options we think is more likely.**Example of a Gamble**• Setup: • First we need to list our options: • B = The US will bomb North Korea by June 30, 2007. • ~B = The US will not bomb North Korea by June 30, 2007. • Second we need to think of some sort of payoff for when one of these options obtains; the payoffs will be understood in terms of gambles, that is, taking one option as more likely than the other.(Here think of a reward for yourself, say getting $10) • Gamble 1: Win $10, if B occurs, otherwise nothing • Gamble 2: Win $10, if ~B occurs, otherwise nothing**There are three possibilities here**• Indifference — You might treat the two options as equally likely, thus you have no reason to accept Gamble 1 over Gamble 2. • You want to take Gamble 1— If you choose Gamble 1, then you feel that B is more likely to occur than ~B. • You want to take Gamble 2— If you choose Gamble 2, then you feel that ~B is more likely to occur than B.**Betting Rates and Payoff Matrixes:**• The examples thus far did not involve taking a risk—in that, you lose nothing if the gamble you chose above did not pay out. • Another way of assigning numbers to confidence levels is to make the decision a risky one, that is, one where the participants stand to lose something by choosing incorrectly.**Betting Rates and Payoff Matrixes**• Notation: • Suppose two people (J and K) make a bet that some event A will turn out. • J bets amount $10on A, and • K bets amount $10 against A. (Betting against A is the same as betting on ¬A.) • What is the total amount of money at stake between J and K? • $20. • The total amount is the sum of the individual bets (i.e., $(10 + 10) ). • The total amount here is called the stake.**Betting Rates and Payoff Matrixes**• In some sense the previous example is fair if the probability is 0.5 • Consider, what would you pay to enter a lottery in which the stake is 10,000 and the chance of winning is 0.1?**Betting Rate**• What we want to know is what a person’s betting rate is. This will be used as a proxy for their personal probability. • The probability that you believe in some hypothesis will be equivalent to your betting on the hypothesis being true. • The betting rate is calculated by taking your individual bet and dividing it by the stake: • Your Betting Rate= Your Bet/Stake • NOTE: We can also determine what a person’s bet is if we know the stake and their betting rate, by the following equation: • Bet = (Personal Betting Rate) * (Stake) • All that we have done here is solved the previous equation for the Bet, which involves multiplying both sides by the Stake.**Bets**• A bet involves two bettors X & Y • X bets £x on H, • Y bets £y on ¬H (i.e. against H). • The winner takes £(x+y) • Thus the odds against H are y:x • The betting rate of X on H is b(X) = x/(x+y)**Definition: Fair Bet**• A bet on H is fair for X iff X is indifferent between the two options of • Betting on H with wager £x; or • Betting against H with wager £y. • A fair bet is one where X ’s betting rates £x/(x+y) and £y/(x+y) are such that X does not prefer to bet either on H or on ¬H at those rates (or, equivalently, X wouldn’t prefer either side at those odds). • A fair bet is not one we would take: in betting we usually wish to win, and hence to bet at unfair rates—unfair in our favour!**Betting Rates and degrees of belief.**• If agent X has betting rate b(H) in what X regards as a fair bet on H, then X ’s degree of belief in H is equal to b(H). • As we deﬁne them, X need not actually bet to have a belief: X must simply have preferences which would make him indifferent between the two betting options, whether or not X ever acts on those preferences (or is ever faced by just those options). • We have completed task one: devised a method for assigning numerical values to propositions.**Pay off Matrix**• Assume X bets on A, where the Stake = £S and X’s betting rate = p. • Then X’s bet is £pS, which is what he stands to lose, he stands to gain • £S(1-p), this can be shown in a pay off matrix: • If you bet for A, then: • (i) If A occurs, you get S(1 - p), or • (ii)If ~A occurs, you get –pS. • If you bet against A, then: • (i) If A occurs, you get - S(1 - p); or • (ii) If ~A occurs, you get pS.**Subjective Belief**• So now we have assigned numbers representing degrees of belief to propositions. • Note these are so far entirely personal (subjective). • Are there any rules governing these beliefs? • Next we shall show that, remarkably, they obey the rules of probability.**Set of Propositions**• First we consider (rather informally) the set of propositions we are dealing with. • These can be any set of propositions, P, providing they satisfy a Boolean algebra: • H in P; implies not H in P • H1 in P and H2 in P; implies (H1 V H2) in P**Ramsey De Finetti Theorem**• We are now in a position to embark on our second task: proving that rational degrees of belief obey the rules of the probability calculus. • That is, we must prove • Theorem (Ramsey De Finetti) If X ’s degrees of belief are rational, then X ’s degrees of belief function defined by fair betting rates is (formally) a probability function. • We begin by settling what it means for degrees of belief to be rational.**Rationality**• Deﬁning ‘rationality’ in general is a huge task. It is extremely difﬁcult, if not impossible, to give an uncontroversial account; it would be unfortunate if we had to give such an account before we could prove our theorem. • Thankfully, we don’t have to. We do accept the following: • If we can show that nonprobabilistic belief would lead to sure betting losses, we can infer from this Commonsense Principle that such belief are not rational.**Sure-Loss Contracts:**• The idea is that when your betting rates are inconsistent, we could think of a crafty person being able to develop a betting contract, by taking your betting rates, where you lose no matter what happens. When you can only be put in a situation where you are sure to lose if you betting rates are inconsistent, and this (as we will see) only happens when your betting rates violate the rules of probability.**Sure loss contract or Dutch Book**• BETTING CONTRACT DEFINED: A betting contract is a contract between two people to settle a bet at some agreed upon set of betting rates. • BOOKMAKER DEFINED: A bookmaker is someone who makes betting contracts. He pays you if you win, and collects from you if you lose. • SURE-LOSS CONTRACT DEFINED: A sure-loss contract is a betting contract in which you will lose no matter what outcome occurs.**Example of Sure Loss Contract:**• Recall from above: • B = The US will bomb North Korea by June 30, 2005. • Your betting rate on B is 5/8. • ~B = The US will not bomb North Korea by June 30, 2005. • Your betting rate on ~B is 3/4 • The bookmaker chooses which way to run the bet, given the betting rates that you have already selected. The bookmaker is a crafty fellow so he will take advantage of both your odds, so you will place independent bets for B and ~B.**The Bets**• Bets on B (at a rate of 5/8): • You bet $5 and the bookmaker bets $3. • Bets on ~B (at a rate of 3/4): • You bet $6 and the bookmaker bets $2.**Result**• What happens if US bomb Korea (i.e., B occurs)? • What do you win? • You would win $3. This is what the bookmaker put up against your bet on B. • What do you lose? • You would lose $6. This is what you bet on ~B. • What is the net payoff if B occurs? • YOU WOULD SUFFER A NET LOSS OF $3. • What happens if US don’t bomb Korea (i.e., ~B occurs)? • What do you win? • You would win $2. This is what the bookmaker put up against your bet on ~B.. • What do you lose? • You would lose $5. This is what you be on B. • What is the net payoff if ~B occurs? • YOU AGAIN WOULD SUFFER A NET LOSS OF $3.**Pay Off Matrix**The basic idea is that, no matter how things turn out, you lose $3. This is what a sure-loss contract looks like, since these were your own betting rates you cannot cry foul that you were mislead by the bookmaker. The notion of a sure-loss contract gives us a way to define coherence among betting rates.**The Argument for Coherence from the Basic Rules of**Probability • Outline: • What we are aiming for is a notion of a set of betting rates being coherent if and only if the set of betting rates satisfies the basic rules of probability. • What we will do is show that when we develop betting rates that violate the rules of probability the result will be that a bookmaker can construct a sure-loss contract from our betting rates. • First • First we show all beliefs must lie within 0 and 1.**Normal Beliefs**• If b(H) is between 0 and 1 it is said to be normal. • We can show that if we do not have belief that is normal then we are vulnerable to a sure loss contract. • It is sufficient to show this for b(H) > 1 as • b(H) < 0 implies b(¬H) > 1.**Normal Beliefs**• If b(H) > 1 then our betting rate is expressible as x/(x+y) where x > (x+y) • hence y is negatitve. • Hence we are indifferent between • Betting x on H and either losing x is H is false; or winning (x+y) if H is true; but (x+y) < x so we lose either way: a sure loss • Betting y on ¬H and either losing y (a win); or winning x+y; either way a sure win. • Being indifferent between a sure loss and a sure win is irrational and the book maker can always make us lose.**Argument for Rule of Addition:**• When A and B are mutually exclusive events, • then Pr(A v B) = Pr(A) + Pr(B) • If our betting rates are to meet this, then our betting for (AvB) should equal the sum of our betting rate for (A) and our betting rate for (B). • Thus, if A and B are mutually exclusive, then the Addition Rule requires: • Betting rate on (A v B) = betting rate on A + betting rate on B**Strategy**• What we will do is see if a sure-loss contract can arise when we violate this rule. • Consider three betting rates: • Betting rate on A = p • Betting rate on B = q • Betting rate on A v B = r • Assume that the restricted rule of addition is violated here. • Thus: r < p + q. That is, assume that we assign a betting rate to r that is less than the sum of the betting rates for p and q. Thus, the sum of p and q will always be greater than r.**Layout of The Bets:**• Suppose a bookmaker asks you to make bets on your rates simultaneously, where the stake for each bet is $1. • We bet $1 dollar so we can use the betting rates as proxies for the bets themselves. • Bet 1: • Here we bet p on A. • What do we win if A occurs? • We win the remainder of the stake, which is (1 – p). • What do we lose if ~A occurs? • We lose our original bet of p. • Bet 2: • Here we bet q on B. • What do we win if B occurs? • Here we win the remainder of the stake, which is (1 – q). • What do we lose if ~B occurs? • Here we lose our original bet q.**Layout of The Bets:**• Bet 3: • Here we bet (1 – r) against (A v B). • Since this is against either A or B, we will only win if neither A nor B occurs. • What do we win if neither A or B occurs? • Here we would win r. • What do we lose if either A or B occurs? • Here we would lose our original bet of (1 – r).**Setting up the Payoff Table:**• Recall that we are assuming that r is less that the sum of p and q, that is, r < p + q. • If that is so, then r – (p + q) is always negative. That is, regardless of what p and q sum to, r must be less than that. Thus, we would lose given these better rates no matter which scenario took place. • Payoff (iii) is the easiest to see: Neither A or B are true. • Here you win r, less p and q. • Therefore, the net payoff is r – p – q. • But since r is less than p and q, this means that regardless of what r is, you lose [r - (p + q)].**Simple Numerical example**• Suppose I have b(A) = 0.2, b(B) = 0.3, b(A v B) = 0.4, hence b¬(A v B) = 0.6. • The book keeper offers a £1 stake on each proposition ¬A and ¬B and (A v B) • If A & ¬B my gain is -0.1 • +0.8 - 0.3 - 0.6 • If B & ¬A my gain is -0.1 • -0.2 + 0.7 -0.6 • If neither A nor B my gain is -0.1 • -0.2 – 0.3 + 0.4**Simple Numerical example**• Suppose I have b(A) = 0.2, b(B) = 0.3, b(A v B) = 0.6 hence b¬(A v B) = 0.4. • The book keeper offers £1 with me taking the bets ¬A and ¬B and (A v B) • If A my gain is -0.1 • -0.2 + 0.7 - 0.6 • If B my gain is -0.1 • +0.8 - 0.3 – 0.6 • If neither A nor B my gain is -0.1 • -0.2 – 0.3 + 0.4**Belief = Betting Rates**• Thus, we have a sure-loss contract, and this means that by violating the restricted rule of addition we end up with inconsistent betting rates. • If our betting rates here had followed the restricted rule of addition, that is, if r = p + q, then there would have been no way to create a sure loss contract involving A, B, and A v B. • LARGER POINT: It is a necessary and sufficient condition that a set of betting rates, including conditional betting rates, should be coherent is that they should satisfy the basic rules of probability.**Conditional Probability**• We have left out conditional probability but this can be easily demonstrated using similar arguments.**So now we have completed our first two tasks:**Assign numbers to measure the strength of a belief; Prove that those numbers must be, formally, probabilities; and Next we need to Provide a rule for updating beliefs given new evidence. Quantifying Belief**Conditional Probability**• We shall next do a little revision on conditional probability as it is necessary to know about this to define Bayes’ rule.**Conditional Probability**• Many times, however, we take the probability of one event to be dependent on another event occurring (or proposition being true). • Example of Conditional Probability Statement: • P2: Given that she starts the chemotherapy immediately, Sarah has a 70% of stopping the spread of the cancer. • Conditional probabilities are complex statements—they are composed of two constituent statements. • In each case here, there are two sentences (or events) that are taken to be related to one another: • The probability of one of the statements is dependent on the other statement being true.**Conditional Probability**• NOTATION FOR CONDITIONAL PROBABILITY: Pr(•/•); • e.g., Pr(A/B) • We can read conditional probabilities in the following ways: • The probability of A given B. • The probability of A on B. • The probability of A on the condition that B. • Definition of Conditional Probability**Conditional Probability**• Definition of Conditional Probability • Read: • When the probability of B is greater than 0, then the probability of A given B is equal to the probability of A and B jointly occurring, divided by the probability of B occurring without A. • Why must B be greater than 0? • Because we cannot divide by 0.**So now we have completed our first two tasks:**Assign numbers to measure the strength of a belief; Prove that those numbers must be, formally, probabilities; and Next we need to Provide a rule for updating beliefs given new evidence. Quantifying Belief