1 / 93

Chapter 8

Chapter 8. Continuous Time Markov Chains. Definition. A discrete-state continuous-time stochastic process is called a Markov chain if for t 0 < t 1 < t 2 < …. < t n < t , the conditional pmf satisfies the relation

jeb
Télécharger la présentation

Chapter 8

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 8 Continuous Time Markov Chains

  2. Definition • A discrete-state continuous-time stochastic process is called a Markov chain if for t0 < t1 < t2 < …. < tn < t , the conditional pmf satisfies the relation • A CTMC is characterized by state changes that can occur at any arbitrary time • Index space is continuous. • The state space is discrete valued.

  3. Continuous Time Markov Chain (CTMC) • A CTMC can be completely described by: • Initial state probability vector for X(t0): • Transition probabilities. • Also,

  4. Homogenous CTMCs • is a time-homogenous CTMC iff Or, the conditional pmf satisfies: • A CTMC is said to be irreducible if every state can be reached from every other state, with a non-zero probability. • A state is said to be absorbing if no other state can be reached from it with non-zero probability.

  5. CTMC Chapman-Kolmogorov Equation • It can also be written as : • In the matrix form, (Matrix Q is called the infinitesimal generator matrix (or simply Generator Matrix)

  6. CTMC Steady-state Solution • Steady state solution of CTMC • Irreducible CTMCs having +ve steady-state {πj} values are called recurrent non-null. • Performance measures may be computed by assigning reward rates to states and computing expected steady state reward rates • Accumulated reward (over an interval of time)

  7. Continuous Time Birth-Death Process • The CTMC and i={0,1,2,…} forms a B-D process, if λi, i={0,1,2,..} and μi, i={1,2,..}exists, and λi: Birth rate (>= 0) and μi: Death rate (>= 0)

  8. Continuous Time Birth-Death Process (contd.) In Steady-state,

  9. Steady State Equations These are called balance eqs. Re-arranging above, = 0

  10. M/M/1 Queue Poisson arrival Process with rate λ • Arrivals follow Poisson distribution, i.e., inter-arrival times are all i.i.d, EXP(λ). • Inter-departuretimes are i.i.d, EXP(μ). • N(t): birth-death proc., λk=λ; μk=μ. • Define, ρ=λ/μ (traffic intensity, in Erlangs)

  11. M/M/1 queue (contd.) • From the balance flow equations, we get • ρ < 1 (for reasons of stability). • Expected # of customers,

  12. M/M/1 queue (contd.) • This measure can be viewed as a weighted average, . • By choosing suitable weights to the states of a CTMC, we can get most measures of interest and the resulting model is known as the MRM(Markov Reward Model). • Other measures: • Average queue length (E[n]) • Average (expected) response time • Average (expected) wait time etc.

  13. M/M/1 queue: Little’s formula • Let the random variable R denote the response time (defined as the time elapsed from the instant of job arrival until its completion) Little’s law states E[R] = E[N]/λ • Here • Response time (R) = wait time (W) + service time (S) E[W] = E[R] – E[S] = 1/μ(1-ρ) - 1/ μ .

  14. Response time distribution (tagged job approach) • Assuming FCFS and steady-state conditions • If there are already n jobs in the system, the next job (N+1)st will experience a response time =R= S*+S’1+S2+..+SN • S* : service time for the (N+1)st job; S’1+: residual service time for job currently undergoing service (#1). • Because of the memory-less property, these times are EXP( ). • Hence, for some N=n, the LST of R is, • Therefore,

  15. M/M/m queue • m-servers service the queue. μ Poisson arrivals (λ)

  16. M/M/m Queue Solution

  17. M/M/m Queue performance measures • Average queue length E[N]: rk= k

  18. M/M/m Queue performance measures • Server utilization: rv M - number of busy servers. For number of customers 0 <= k <= m, the number of busy servers = k. Beyond that the number of busy servers = m. • A customer may have to join the queue.

  19. Poisson stream behavior • M/M/m: input/output both form Poisson streams. • m=2 case • Case 1: Two independent queues • Case 2: M/M/2 case Two separate Poisson streams  2 separate M/M/1 queues Two separate Poisson streams Combined Poisson steams

  20. Comparative performance • Case 1: For each M/M/1 queue, • Case 2: Common queue M/M/2

  21. M/M/1/n Queue • Finite queue size, finite buffer space  finite state space. • Steady State Solution:

  22. M/M/1/n Queue Performance Measures • Mean queue length (expected # of jobs in the system). • rk = k, • Loss probability • rn = 1, rk = 0, k=0,1,..,n-1 • Throughput • rk =m , k=1,2, ..,n; r0 = 0 (or, rk =l , k=0,1,2, ..,n-1; rn = 0)

  23. M/M/1/n: Response time distribution • Response time distribution: Job may be rejected (or accepted) • Unconditional • Conditional (conditioned on the job being accepted): • Reward assignment: for the kth state, response time experienced by the tagged task is sum of k-service times, each of which is EXP(μ), i.e., k-stage Erlang. • Unconditional • Conditional

  24. Special cases of Birth-Death Process • Pure birth processes • Poisson process • Software Reliability Growth Model: NHPP • Number of software failures occurring in (0, t] is N(t), and N(t) is Poisson with, λ(t) = abe-btand m(t) = E[N(t)] = a(1- e-bt) • Instantaneous failure intensity, λ(t) = b[a-m(t)] • Transient solution may be found using Laplace transforms • Pure death processes • No-repairs

  25. Markov Availability Model

  26. UP 1 DN 0 2-State Markov Availability Model 1) Steady-state balance equations for each state: • Rate of flow IN = rate of flow OUT • State1: • State0: 2 unknowns, 2 equations, but there is only one independent equation.

  27. 2-State Markov Availability Model(Continued) Need an additional equation: Downtime in minutes per year = * 8760*60

  28. 2-State Markov Availability Model(Continued) 2) Transient Availability for each state: • Rate of buildup = rate of flow IN - rate of flow OUT This equation can be solved to obtain assuming P1(0)=1

  29. 2-State Markov Availability Model(Continued) 3) 4) Steady State Availability:

  30. Using SHARPE to Solve the models

  31. Markov availability model • Assume we have a two-component parallel redundant system with repair rate . • Assume that the failure rate of both the components is . • When both the components have failed, the system is considered to have failed.

  32. Markov availability model(Continued) • Let the number of properly functioning components be the state of the system. The state space is {0,1,2} where 0 is the system down state. • We wish to examine effects of shared vs. non-shared repair.

  33. Markov availability model(Continued) 2 1 0 Non-shared (independent) repair 2 1 0 Shared repair

More Related