SENG 521 Software Reliability & Testing

1. SENG 521Software Reliability & Testing Software Reliability Models (Part 2a) Department of Electrical & Computer Engineering, University of Calgary B.H. Far （far@enel.ucalgary.ca） http://www.enel.ucalgary.ca/~far/Lectures/SENG521/02a/ far@enel.ucalgary.ca

2. Contents • Basic Features of the Software Reliability Models • Single Failure Model • Reliability Growth Model • Exponential Failure Class Models • Weibull and Gamma Failure Class Models • Infinite Failure Category Models • Bayesian Models • Early Life-Cycle Prediction Models far@enel.ucalgary.ca

3. Section 1 Basic Features far@enel.ucalgary.ca

4. Goal • It is important to be able to • Predict probability of failure of a component or system • Estimate the mean time to the next failure • Predict number of (remaining) failures during the development. • Such tasks are the target of the reliability management models. far@enel.ucalgary.ca

5. Reliability Model Fault removal: Failure discovery (e.g., extent of execution, operational profile) Quality of repair activity Fault introduction: Characteristics of the product (e.g., program size) Development process (e.g., SE tools and techniques, staff experiences, etc.) Reliability Model Environment far@enel.ucalgary.ca

6. Two Reliability Questions Single failure specification: • What is the probability of failure of a system (or a component)? Multiple failure specification: • If a system (or a component) fails at time t1, t2, …, ti-1, what is the probability of its failure at time ti? far@enel.ucalgary.ca

7. Model Classification • Time domain: Calendar or execution time • Category: Number of failures is finite or infinite. • Type: Distribution of the number of failures experienced by the time specified. • Class (only finite category): Functional form of the failure intensity over time. • Family (only infinite category): Functional form of the failure intensity in terms of the expected number of failures experienced. far@enel.ucalgary.ca

8. Various Reliability Models /1 • Exponential Failure Class Models • Jelinski-Moranda model (JM) • Nonhomogeneous Poisson Process model (NHPP) • Schneidewind model • Musa’s Basic Execution Time model (BET) • Hyperexponential model (HE) • Others far@enel.ucalgary.ca

9. Various Reliability Models /2 • Weibull and Gamma Failure Class Models • Weibull model (WM) • S-shaped Reliability Growth model (SRG) • Infinite Failure Category Models • Duane’s model • Geometric model • Musa-Okumoto Logarithmic Poisson model far@enel.ucalgary.ca

10. Various Reliability Models /3 • Bayesian Models • Littlewood-Verrall Model • Early Life-Cycle Prediction Models • Phase-based model far@enel.ucalgary.ca

11. Another Categorization /1 • Time between failure models • Jelinski-Moranda (Exponential Failure Model) • Musa-Basic (Exponential FailureModel) • NHPP (Exponential FailureModel) • Geometric (Infinite FailureModel) • Musa-Okumoto (Infinite FailureModel) • Littlewood-Verrall (Bayesian Model) far@enel.ucalgary.ca

12. Another Categorization /2 • Failure count models • Generalized Poisson • Shick-Wolverton • Yamada S-shaped • NHPP (Exponential FailureModel) • Schneidewind (Exponential FailureModel) far@enel.ucalgary.ca

13. Section 2 Single Failure Model far@enel.ucalgary.ca

14. f t T f t T Hardware Reliability Models • Uniform model: • Probability of failure is fixed. • Exponential model: • Probability of failure changes exponentially over time far@enel.ucalgary.ca

15. Single Failure Model /1 • Probability Density Function (PDF): depicting changes of the probability of failure up to a given time t. • A common form of PDF is exponential distribution • Usually we want to know how long a component will behave correctly before it fails, i.e., the probability of failure from time 0 up to a given time t. far@enel.ucalgary.ca

16. Single Failure Model /2 • Cumulative Density Function (CDF): depicting cumulative failures up to a given time t. • For exponential distribution, CDF is: far@enel.ucalgary.ca

17. Single Failure Model /3 • Reliability function (R): depicting a component functioning without failure until time t. • For exponential distribution, R is: far@enel.ucalgary.ca

18. Single Failure Model /4 • What is the expected value of failure time T? • It is the mean of the probability density function (PDF), named mean time to failure (MTTF) • For exponential distribution, MTTF is: far@enel.ucalgary.ca

19. Single Failure Model /5 • Median time to failure(tm): a point in time that the probability of failure before and after tm are equal. • Failure Rate z(t): Probability density function divided by reliability function. For exponential distribution, z(t) is: λ far@enel.ucalgary.ca

20. Single Failure Model /6 • System Reliability: is the multiplication of the reliability of its components. • For exponential distribution: far@enel.ucalgary.ca

21. Single Failure Model /7 • System Cumulative Failure Rate: is the sum of the failure rate of its components. • For exponential distribution: far@enel.ucalgary.ca

22. Section 3 Reliability Growth Model far@enel.ucalgary.ca

23. Reliability Growth Models /1 • One can assume that the probability of failure (probability density function, PDF) for all failures are the same (e.g., replacing the faulty hardware component with an identical one). • In software, however, we want to “fix” the problem, i.e., have a lower probability of failure after a repair (or longer Δti = ti-ti-1). Therefore, we need a model for reliability growth (i.e., reliability change over time). far@enel.ucalgary.ca

24. Reliability Growth Models /2 • Common software reliability growth models are: • Basic Exponential model • Logarithmic Poisson model • The basic exponential model assumes finite failures (ν0) in infinite time. • The logarithmic Poisson model assumes infinite failures. far@enel.ucalgary.ca

25. Revision Period 1 Revision Period 4 Validity of the Models • Software systems are changed (updated) many times during their life cycle. • The models are good for one revision period rather than the whole life cycle. far@enel.ucalgary.ca

26. Reliability Growth Models /3 • Parameters involved in reliability growth models: • Failure intensity (λ): number of failures per natural or time unit. • Execution time (τ): time since the program is running. Execution time may be different from calendar time. • Mean failures experienced (μ): mean failures experienced in a time interval. far@enel.ucalgary.ca

27. Reliability Growth Models /4 • Mean failures experienced (μ) for a given time period (e.g., 1 hour execution time) is calculated as: far@enel.ucalgary.ca

28. Reliability Growth Models /5 • Failure intensity (λ) versus execution time (τ) far@enel.ucalgary.ca

29. Reliability Growth Models /6 • Failure intensity (λ) versus mean failures experienced (μ) far@enel.ucalgary.ca

30. Reliability Growth Models /7 • Mean failures experienced (μ) versus execution time (τ) far@enel.ucalgary.ca

31. Reliability Growth Models /8 far@enel.ucalgary.ca

32. How to Use the Models? • Release criteria: time required to test the system to reach a target failure intensity: far@enel.ucalgary.ca

33. Reliability Metrics • Mean time to failure (MTTF): Usually calculated by dividing the total operating time of the units tested by the total number of failures encountered (assuming that the failure rate is constant). • Example: • MTTF for Windows 2000 Professional is 2893 hours or 72 forty-hour workweeks. • MTTF for Windows NT Workstation is 919 hours or 23 workweeks. • MTTF for Windows 98 is 216 hours or 5 workweeks. • Mean time to repair (MTTR): mean time to repair a (software) component. • Mean time between failures (MTBF): MTBF = MTTF + MTTR far@enel.ucalgary.ca

34. Reliability Metrics: Availability • Software System Availability (A): λ is failure intensity tm is downtime per failure • Another definition of availability: • Example: If a product must be available 99% of time and downtime is 6 min, then λ is about 0.1 failure per hour (1 failure per 10 hours) and MTTF=594 min. far@enel.ucalgary.ca

35. Reliability Metrics: Reliability • Software System Reliability (R): λis failure intensity R is reliability t is natural unit (time, etc.) • Example: for λ=0.001 or 1 failure for 1000 hours, reliability (R) is around 0.992 for 8 hours of operation. far@enel.ucalgary.ca

36. far@enel.ucalgary.ca