Probability Theory

1. Probability Theory Part 1: Basic Concepts

2. Sample Space - Events • Sample Point • The outcome of a random experiment • Sample Space S • The set of all possible outcomes • Discrete and Continuous • Events • A set of outcomes, thus a subset of S • Certain, Impossible and Elementary

3. Set Operations • Union • Intersection • Complement • Properties • Commutation • Associativity • Distribution • De Morgan’s Rule S

4. Axioms If If A1, A2, … are pairwise exclusive Corollaries Axioms and Corollaries

5. Computing Probabilities Using Counting Methods • Sampling With Replacement and Ordering • Sampling Without Replacement and With Ordering • Permutations of n Distinct Objects • Sampling Without Replacement and Ordering • Sampling With Replacement and Without Ordering

6. Conditional Probability • Conditional Probability of event A given that event B has occurred • If B1, B2,…,Bn a partition of S, then (Law of Total Probability) S B1 B2 A B3

7. Bayes’ Rule • If B1, …, Bn a partition of S then Example Which input is more probable if the output is 1? A priori, both input symbols are equally likely. input 0 1 1-p p output 0 1 0 1 1-ε ε ε 1-ε

8. Event Independence A B • Events A and B are independentif • If two events have non-zero probability and are mutually exclusive, then they cannot be independent 1 1 ½ ½ C 1 ½ 1 1 ½ ½ 1

9. Sequences of Independent Experiments E1, E2, …, Ej experiments A1, A2, …, Aj respective events Independent if Bernoulli Trials Test whether an event A occurs (success – failure) What is the probability of k successes in n independent repetitions of a Bernoulli trial? Transmission over a channel with ε = 10-3 and with 3-bit majority vote Sequential Experiments