SENG 521 Software Reliability & Testing

SENG 521Software Reliability & Testing Software Reliability Models (Part 2b) Department of Electrical & Computer Engineering, University of Calgary B.H. Far （far@enel.ucalgary.ca） http://www.enel.ucalgary.ca/~far/Lectures/SENG521/02b/ far@enel.ucalgary.ca

Contents • Basic Features of the Software Reliability Models • Single Failure Model • Reliability Growth Model • Exponential Failure Class Models • Weibull and Gamma Failure Class Models • Infinite Failure Category Models • Bayesian Models • Early Life-Cycle Prediction Models far@enel.ucalgary.ca

Goal • It is important to be able to • Predict probability of failure of a component or system • Estimate the mean time to the next failure • Predict number of (remaining) failures during the development. • Such tasks are the target of the reliability management models. far@enel.ucalgary.ca

BackgroundRandomness & Probability far@enel.ucalgary.ca

Randomness /1 • Random actions: • Introduction of defects into the code • Execution of the test-cases • We should define some random processes to represent the randomness • How to handle randomness? • Collect failure data through testing • Find a distribution function that is a best-fit for the collected data • Make assumptions about the presence of errors and reliability far@enel.ucalgary.ca

Randomness /2 • What is a random variable? • A random variable x on a sample space S is a rule that assigns a numerical value to each outcome of S (a function of S into a set of real numbers) • In reliability modeling what can be represented by random variable? • Time of failure • Number of failures far@enel.ucalgary.ca

Probability Distribution /1 • Suppose that a random variable X assigns a finite number of values to a sample space S • Then X induces a distribution functionf that assigns probabilities to the points in Rx Rx ={x1, x2, x3, …, xn} f(xk) = P( X=xk) far@enel.ucalgary.ca

Probability Distribution /2 • The set of ordered pairs [xk, f(xk)] is usually represented by a table or a graph (histogram) • The expected value of X, denoted by E(X) is defined by E(X) = x1 f(x1) + x2 f(x2) + …+ xn f(xn) far@enel.ucalgary.ca

Probability Distribution /3 • Let M(t) be a random process representing the number of failures at time t • The mean function μ(t)represents the expected number of failures at time t μ(t) = E(M(t)) • Failure intensity is the rate of change of the expected number of failures with respect to time λ(t) = d μ(t) / dt • (t) is the number of failures per unit time • (t) is an instantaneous value far@enel.ucalgary.ca

Probability Distribution /4 • Discrete distributions: • Binomial distribution • Poisson distribution • Continuous distributions: • Normal / Gaussian distribution • Lognormal distribution • Weibull distribution • Rayleigh distribution • Exponential distribution • Gamma distribution • 2 distribution far@enel.ucalgary.ca

Jakob Bernoulli (1645-1705) Binomial Distribution /1 • Gives probability of exact number of successes in n independent trials, when probability of success p on single trial is a constant. • Situations with only 2 outcomes (success or failure) • Probability remains the same for all independent trials (Bernoulli trials) far@enel.ucalgary.ca

Binomial Distribution /1 • Probability of exactly x successes: n : number of trials f(x) : probability of x successes in n trials p : probability of success q : probability of failure p + q =1 far@enel.ucalgary.ca

Binomial Distribution /2 • Probability of at least r successes: n : number of trials f(x) : probability of x successes in n trials p : probability of success q : probability of failure p + q =1 F(r) : probability of obtainingr or fewer successes in n trials Calculating F(r) becomes increasingly difficult as n (sample set) gets larger It is possible to find an approximate solution by means of a normal distribution far@enel.ucalgary.ca

Example: Binomial Distribution • Acceptance sampling: • A lot is accepted if not more than 2 defectives are found in a sample of 6. The defect probability is 25%. • Probability of having exactly 2 defects in the lot is: • Probability of having more than 4 defects in the lot is: far@enel.ucalgary.ca

Simeon Poisson (1781-1840) Poisson Distribution /1 • Some events are rather rare, they don’t happen that often. Still, over a period of time, we want to say something about the nature of rare events. • Poisson distribution is special case of binomial distribution (either p or q is very small and n very large) • Conditions under which a Poisson distribution holds • counts of rare events • all events are independent • average rate does not change over the period of interest far@enel.ucalgary.ca

Poisson Distribution /2 • Poisson distribution is a special case of binomial distribution (either p or q is very small and n very large): =np • : mean rate of occurrence (in statistics literature is usually denoted by ) • x : observed number of failures far@enel.ucalgary.ca

Example: Poisson Distribution • Suppose that the defect rate is only 2% find the probability that there are 3 defective items in a sample of 100 items. far@enel.ucalgary.ca

Exponential Failure Time Models far@enel.ucalgary.ca

Exponential Failure Time Models • Assumes finite failure models • Functional form of failure intensity function is exponential • Binominal type: • All faults have a constant hazard rate z(t) = • Hazard rate for the i-th fault is a function of the remaining number of faults • Failure intensity is far@enel.ucalgary.ca

Various Reliability Models • Exponential Failure Time Models • Jelinski-Moranda model (JM) • Nonhomogeneous Poisson Process model (NHPP) • Schneidewind model • Musa’s Basic Execution Time model (BET) • Hyperexponential model (HE) • Others far@enel.ucalgary.ca

Jelinski-Moranda Model /1 • Jelinski-Moranda model follows exponential distribution but differs from the Geometric model in that the parameter used is proportional to the remaining number of faults rather than a constant. • All faults equally contribute to the reliability of the system Hazard Rate z(t): Probability density function divided by reliability function.  is a constant far@enel.ucalgary.ca

Jelinski-Moranda Model /1 • Assumptions: • The rate of fault detection is proportional to the current fault content of the software. • The fault detection rate remains constant over the intervals between faults. • A fault is corrected instantaneously without introducing new faults into the software. • Every fault has the same chance of being encountered within a severity class as any other fault in that class. • The failures are independent. • Data requirement: • Either actual failure times: t1, t2, …, tn • Or elapsed time between failures: xi = ti – ti-1 far@enel.ucalgary.ca

Jelinski-Moranda Model /2 • Meantime between failures at time t MTTF = 1/  (N-(i-1)) • Mean failures experienced • Failure intensity function N total number of faults (constant) Parameters to estimate using collected data are: N and  far@enel.ucalgary.ca

Non Homogeneous Poisson Process Model /1 • Assumes that the cumulative number of failures detected at any time follows a Poisson distribution. • Time periods can be unequal • Data requirement: • Fault counts on each testing interval: f1, f2, …, fn • Completion time of each period: t1, t2, …, tn far@enel.ucalgary.ca

Non Homogeneous Poisson Process Model /2 • Assumptions: • Every failure has the same chance of being detected as any other failure. • The cumulative number of failures detected at any time follows a Poisson distribution with mean (t). • That mean is such that the expected number of failures in any small time interval about t is proportional to the number of undetected failures at time t. • The mean is assumed to be a bounded non-decreasing function with (t) approaching in the limit, as the length of testing goes to infinity. • Perfect debugging is assumed. far@enel.ucalgary.ca

Non Homogeneous Poisson Process Model /3 • Mean failures experienced • Failure intensity function Parameters to estimate using collected data are: N and b far@enel.ucalgary.ca

Schneidewind Model • Assumes that the current fault rate might be a better predictor of the future behaviour than the observed rate in the distant past • Suppose that there are n units of time all of fixed length • Has 3 variations: • Utilize all individual fault counts from all n time periods • Ignore fault counts from 1 to s-1 periods (forget the distant past) • Use cumulative fault count for 1 to s-1 periods and individual fault counts for s to n far@enel.ucalgary.ca

Musa Basic Model /1 • One of the most widely used software reliability models. • Uses execution time rather than calendar time. 0 and 1 are parameters. 0 is equal to the number of faults in the system and 1 is a fault reduction factor. far@enel.ucalgary.ca

Musa Basic Model /2 • Assumptions: • The detections of failures are independent of one another. • The software failures are observed (i.e. the total number of failures has an upper bound). • The execution times (measured in CPU time) between failures are piecewise exponentially distributed. • The hazard rate is proportional to the number of faults remaining in the program. • The fault correction rate is proportional to the failure occurrence rate. • Perfect debugging is assumed. far@enel.ucalgary.ca

Musa Basic Model /3 • Meantime between failures at time t • Mean failures experienced • Failure intensity function n0 = total number of errors in the program MTTF0=MTTF at the start of testing C= test compression factor far@enel.ucalgary.ca

Musa Basic Model /4 • Number of errors at time t • Reliability n0 = total number of errors in the program MTTF0=MTTF at the start of testing C= test compression factor far@enel.ucalgary.ca

Hyper-exponential Model /1 • The basic idea is that the different sections (or classes) of the software experience an exponential failure rate; however, the rates vary over these sections to reflect their different natures. This could be due to different programming groups doing the different parts, old versus new code, sections written in different languages, etc. • We thus reflect the sum of these different exponential growth curves, not by another exponential, but by a hyper-exponential growth curve. far@enel.ucalgary.ca

Hyper-exponential Model /2 • Data requirement: • Fault counts on each testing interval: f1, f2, …, fn • Completion time of each period: t1, t2, …, tn • Failure intensity function far@enel.ucalgary.ca

far@enel.ucalgary.ca

SENG 521 Software Reliability & Testing