1 / 60

A brief review

A brief review. The exponential distribution. The memoryless property.  Exponentially distributed random variables are memoryless. The exponential distribution is the only distribution that has the memoryless property. The minimum of n exponentially distributed random variables.

thuong
Télécharger la présentation

A brief review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A brief review

  2. The exponential distribution

  3. The memoryless property

  4.  Exponentially distributed random variables are memoryless.

  5. The exponential distribution is the only distribution that has the memoryless property.

  6. The minimum of n exponentially distributed random variables Suppose that X1, X2, ..., Xn are independent exponential random variables, with Xi having rate li, i=1, ..., n. What is P(min(X1, X2, ..., Xn )>x)?

  7. Comparing two exponentially distributed random variables Suppose that X1 and X2 are independent exponentially distributed random variables with rates l1 and l2. What is P(X1 < X2)?

  8. The Poisson process The counting process {N(t) t ≥ 0} is said to be a Poisson process having rate l, l > 0, if (i)N(0) = 0. (ii) The process has independent increments. (iii) The number of events in any interval of length t is Poisson distributed with mean lt. That is for all s, t ≥ 0

  9. The distribution of interarrival times for a Poisson process Let Tn denote the inter-arrival time between the (n-1)th event and the nth event of a Poisson process, then the Tn (n=1, 2, ...) are independent, identically distributed exponential random variables having mean 1/l.

  10. Continuous Time Markov Chains (CTMC)

  11. A CTMC is a continuous time analog to a discrete time Markov chain • A CTMC is defined with respect to a continuous time stochastic process {X(t): t ≥0} • If X(t) = i the process is said to be in state i at time t • {i: i=0, 1, 2, ...} is the state space

  12. A stochastic process {X(t): t ≥0} is a continuous time Markov chain if for all s, t , u ≥0 and 0 ≤ u < s • P{X(t+s)=j|X(s)=i, X(u)=x(u), 0 ≤ u < s} = P{X(t+s)=j|X(s)=i}

  13. A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t).

  14. A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t). •  Pij(t) = P{X(t+s)=j|X(s)=i}

  15. A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t). •  Pij(t) = P{X(t+s)=j|X(s)=i} • Note: We shall always assume that the stationary property holds

  16. Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable).

  17. Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable). • Example 1:X(0)=2, the first three transitions occur at t1 =3, t1 = 4.2 and t1 = 6.5 and X(3)=4, X(4.2) = 2 and X(6.5) =1.

  18. Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable). • Example 1:X(0)=2, the first three transitions occur at t1 =3, t2 = 4.2 and t3 = 6.5 and X(3)=4, X(4.2) = 2 and X(6.5) =1. •  The first sojourn time in state 4 = 4.2-3 = 1.2 • The second sojourn time in state 2 = 6.5 – 4.2 = 1.3

  19. Example 2: Suppose the process has been in state 3 for 10 minutes, what is the probability that it will not leave state 3 in the next 5 minutes.

  20. Example 2: Suppose the process has been in state 3 for 10 minutes, what is the probability that it will not leave state 3 in the next 5 minutes. • P(T3 > 15|T3> 10) = P(T3 > 5)

  21. More generally, • P(Ti > s+t|Ti> s) = P(Ti > t) •  Ti is memoryless and therefore has the exponential distribution

  22. An alternative definition of a CTMC A CTMC is a stochastic process having the properties that each time it enters a state i (i) the amount of time it spends in that state before making a transition into a different state is exponentially distributed with some mean 1/vi (or transition rate vi ) (ii) when the process leaves state i, it next enters state j with some probability Pij(Pii=0 and SjPij=1, for all i)

  23. A CTMC is a stochastic process that moves from state to state according to a probability transition matrix (similar to a discrete time Markov chain) , but the amount of time it spends in each state is exponentially distributed.

  24. To define a CTMC, we need to define a state space, a probability transition matrix, and a set of transition rates.

  25. Example 1: Customers arrive to a store according to a Poisson process with rate l. Let N(t) be the total number of customers that have arrived by time t.

  26. Example 1: Customers arrive to a store according to a Poisson process with rate l. Let N(t) be the total number of customers that have arrived by time t.  State space is {0, 1, 2, ...}; Ti is exponentially distributed with mean 1/l Pij =1 if j=i+1 and Pij = 0 otherwise

  27. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.

  28. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.  State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ...

  29. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.  State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ... The above is an example of an M/M/1 queue. The M/M/1 queue is an example of a birth and death process.

  30. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.  State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ... The above is an example of an M/M/1 queue. The M/M/1 queue is an example of a birth and death process.

  31. Birth and death process Example 3: Customers arrive to a service center according to a Poisson process with rate lnwhen there are n customers in the system. Customers take an amount Tn that is exponentially distributed with mean 1/mn when there are n customers in the system.

  32. Birth and death process Example 3: Customers arrive to a service center according to a Poisson process with rate lnwhen there are n customers in the system. Customers take an amount Tn that is exponentially distributed with mean 1/mn when there are n customers in the system.  State space is {0, 1, 2, ...} P0,1 = 1; Pn,n+1=ln/(ln+mn); Pn,n-1=mn/(ln+mn) T0 =1/l0v0 =l0; Tn = 1/(ln +mn) vn =ln+mnfor n =1, 2, ...

  33. State transition diagrams for a B&D process l0 l1 l2 0 1 2 3 m1 m2 m3

  34. The Poisson process is a birth and death process with rate ln=l and mn=0. • The M/M/1 queue is described by a birth and death process with rate ln=l and mn=m.

  35. Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.

  36. Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.  The system can be modeled as a birth and death process with transition rates ln = l; mn = nm if 1 ≤ n < m mn = mm if n ≥ m

  37. Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.  The system can be modeled as a birth and death process with transition rates ln = l; mn = nm if 1 ≤ n < m mn = mm if n ≥ m The above is an example of an M/M/m queue.

  38. Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i)  qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)

  39. Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i)  qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)  vi =Sj viPij= Sj qij Pij =qij/vi =qij/Sj qij

  40. Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i)  qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)  vi =Sj viPij= Sj qij Pij =qij/vi =qij/Sj qij Specifying the instantaneous transition rates determines the parameters of the CTMC

  41. State transition diagrams q0,2 q0,1 q1,2 0 1 2 q2,0 q2,1 q2,0

  42. Properties It can also be shown that

  43. The Chapman-Kolmogrov equations

  44. The Chapman-Kolmogrov equations

  45. The Chapman-Kolmogrov equations

  46. The Chapman-Kolmogrov equations

  47. The Chapman-Kolmogrov equations

  48. The Chapman-Kolmogrov equations

  49. Kolmogrov’s backward equations

  50. Kolmogrov’s backward equations

More Related