Quality of Information Systems

1. Qualityof Information Systems Introduction

2. Software is Everywhere Quality of Software is Important !

3. Ariane_5 • modern European rocket • complicated design • exploded on maiden flight after 37 sec. • project costs M$ 7; direct costs M$ 2.5 • error in software of horizontal positioning system : • overflow of value, • leading to exception, • leading to self destruction, • back-up system did exactly the same …… • software copied from Ariane-4 • incomplete testing procedures

4. Mars Climate Orbiter (1999) • Mission: • Mars surface climate measurements • 5 year relay-station Mars Polar Lander (2001) • Crashed when landing on Mars surface. • Errors in software: • failed English units to metrics conversion, • faults in computer models • Errors in process management: • too informal communication channels between engineering groups • interconnecting aspects checking not robust • inadequate verification/validation requirements between project groups and project/prime contractor (http://mars.jpl.nasa.gov/msp98/orbiter/)

5. WWMCCS incident (1979) • World Wide Monitoring Command and Control System computer network • At November 9 1979 it reports missile attack from Soviet Union aimed toward United States • Attack actually was a simulation! • Errorsunknown, but likely causes in software: • system could not differentiate simulation / reality, or • user interface inadequate in presenting information

6. The theme of NIII is: Quality of Software What is Quality ?

7. “A program reads three integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is scalene, isosceles, or equilateral.” What is Quality ? What is important for the quality of this program?

8. What is Quality ? ISO 9126 (Software Product Quality) : Quality is the totality of characteristics of an entitythat bear on its abilityto satisfy stated and implied needs

9. What is Quality ? Quality : Characteristics of product leading to satisfying stated and implied needs • Classification of quality characteristics ? • How to specify quality and needs ? • stated / explicit needs and requirements • implied / implicit needs and wishes • legal requirements • Is quality always yes / no ? • How to quantify quality ? • How to measure quality ?

10. Classification of Software Quality • functional testing • reliability testing • usability testing • performance testing • maintainability testing ?? • portability testing ? Characteristics according to ISO 9126 : • Functionality • suitability accuracy, security, compliance, interoperability • Reliability • maturity, fault tolerance, recoverability • Usability • understandability, learnability, operability • Efficiency • time behaviour, resource utilization • Maintainability • Analysability, changeability, stability, testability • Portability • Adaptability, intallability, conformance, replaceability

11. Non-Functional Quality • In the old days . . . . when computers were operated by experts,running in batch mode, over-night, stand-alone,quality mainly involved : checking whether the program does what is should do functional quality characteristics • But in these days . . . .now that everybody uses computers (PC, video, gsm, shaver, ...),interactively, on-line, connected to the world,we want more : user-friendliness, speed, robustness, security, ……  non-functional quality characteristics

12. Sorts of Quality • Quality characteristics: • …… • Who: • user perceived quality, • operational quality, • certification quality, • …… • Phase • specification quality • design quality • test quality • ……

13. requirements acceptancetest systemtest detaileddesign moduletest specification unittest implementationcode Quality Checking in the Development Process : The V-Model

14. requirements acceptancetest systemtest detaileddesign moduletest specification unittest implementationcode Quality in the Development Process users

15. spec build accept exploit & use destroy deliver Software Quality • Testing as measurement for software quality • Software quality as quantity to predict future operation : • reliability, dependability, MTBF • expected value, expected costs, probability of failure • risk = probability of failure  expected cost of failure but software reliability theory is immature ! systemtest acceptancetest quality ?

16. mtbf-1 time Hardware Quality Hardware quality • physical failures • product changes with time • mass production • statistical measurements Software quality • design quality • no change with time,except by human intervention MTBF models for software ??

17. How to Obtain Quality ? • Quality management: • which actions contribute how to quality • Well organized software process • requirements management • design • traceability • configuration management • …… • Testing, validation, verification • Well educated people • Appropriate techniques and tools • …… • And then: How to check that quality reqs. are satisfied ?

18. No Silver Bullet …… requirement management quality control testing software process verification reviewing inspection CMM static analysis certification debugging walk-through QUALITY

19. developmentprocess Software Quality rawmaterials people For quality in software product : • Prescribe methods & techniques • IEC 61508 • Certify tools • certified ADA compiler • Educate and certify people • RI, Certified Tester • Check inputs / raw materials • Process control • ISO 9000, CMM, SPI • Product assessment : • testing • operational feedback methods &techniques tools softwareproduct

20. Software Quality in Organizations • Software product quality • Testing as one of the methods to measure quality • Project quality • Well-defined development cycle • Integrated quality plan (V & V plan) • Feedback from testers to developers • Organization quality • Standard development process • Standard and managed quality procedures • Supported (test & validation) techniques • Learn from previous projects

21. Quality by Testing • No faults detected : Cheers !Program is OK ! • Or: Was the test NOT OK ? • No errors faults detected  there are no faults • How to determine the quality of test suites ?And indirectly the quality of tested products ? • Meyers [1978] “The purpose of testing is to find errors”Now: “…… measuring quality ……”

22. Verification & Validation • Verification : building the product right • concerns development process • intermediate products • e.g. checking the testability of the specification • Validation : building the right product • whether final product meets requirements and needs • e.g. by reviewing the specification

23. requirements specifying validation verification specification/ design implementing testing systemimplementation Traditional Development Process

24. Use mathematics to model relevant parts of software Precise, formal semantics: no room for ambiguity or misinterpretation Allow formal validation and reasoning about systems Amenable to tools: automation Examples: Z Temporal Logic SDL LOTOS Promela Labelled Transition Systems ...... Formal Methods

25. informalrequirements formalmodelling formal verification,simulation formalvalidation formalmodel implementing formaltesting systemimplementation Formal Development Process

26. Verification : formal manipulation prove properties performed on model Testing : experimentation show error concrete system Formal Verification and Testing formal world concrete world Verification is only as good as the validity of the model on which it is based Testing can only show the presence of errors, not their absence

27. Errors, Faults, Failures Failure incorrect behaviour of the IUT Fault problem in the program causing the failure Defect detected failure Issue politically correct term for failure ( before cause is determined ) Error can only be made by a human being Bug fault

28. Other Lectures : Overview • Quality characteristics • Quality in the software development process • Software Process Improvement / CMM • Formal methods • Testing • Standards and certification • Quality Assurance