Software Engineering: Software Quality Assurance

1. Software Engineering:Software Quality Assurance Romi Satria Wahonoromi@romisatriawahono.nethttp://romisatriawahono.net+6281586220090

2. Romi Satria Wahono • SD Sompok Semarang (1987) • SMPN 8 Semarang (1990) • SMA Taruna Nusantara, Magelang (1993) • S1, S2 dan S3 (on-leave)Department of Computer SciencesSaitama University, Japan (1994-2004) • Research Interests: Software Engineering,Intelligent Systems • Founder danKoordinatorIlmuKomputer.Com • Peneliti LIPI (2004-2007) • Founder dan CEO PT Brainmatics Cipta Informatika

3. Course Contents-1- • Introduction to Software Engineering • WhatisSoftware • WhatisSoftware Engineering • Discipline andCurriculumof Software Engineering • Software Engineering Profession • Profession, Ethics and Certification • Software Industry and Market • Internet Business Model and Trends

4. Course Contents-2- • Software Engineering Process • Software Development Life Cycle (SDLC) • Software Development Methodologies • Software Development Notation (UML) and Tools • Object-Oriented Paradigm • Software Construction • Software Construction Process • Estimating the Size of Software Project

5. Course Contents-3- • Software Quality Assurance • The Uniqueness of SoftwareQualityAssurance • What is Software Quality • Software Quality Factor • Software Testing • Software Engineering Research • ComputingResearch Methodology • Research Trendsin Software Engineering • Case Study: Developing Research Proposal in Software Engineering Field

6. Software Quality Assurance

7. Contents • The Uniquenessof SoftwareQualityAssurance • What is Software Quality • Software Quality Factor • Software Testing

8. 1. The Uniqueness of Software Quality Assurance

10. Software vsOther Industrial Products Product Complexity Product Visibility Product Development Process

12. The Uniqueness of theSoftware Development

13. Warranty Lawsuits Mortensonvs. Timeberline Software (≈1993) Mortenson used a TS application when creating a bid to build a hospital The software created a bid that was $2M too low TS knew about the bug, but had not sent an update to Mortenson The State of Washington Supreme Court ruled in favor of TS

14. Warranty Laws Article 2 of the Uniform Commercial Code Uniform Computer Information Transaction Act (UCITA)allows software manufacturers to: (≈1999) disclaim all liability for defects prevent the transfer of software from person to person remotely disable licensed software during a dispute does not apply to embedded systems

15. Disclaimer of Warranties DISCLAIMER OF WARRANTIES. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, MICROSOFT AND ITS SUPPLIERS PROVIDE TO YOU THE SOFTWARE COMPONENT, AND ANY (IF ANY) SUPPORT SERVICES RELATED TO THE SOFTWARE COMPONENT ("SUPPORT SERVICES") AS IS AND WITH ALL FAULTS; AND MICROSOFT AND ITS SUPPLIERS HEREBY DISCLAIM WITH RESPECT TO THE SOFTWARE COMPONENT AND SUPPORT SERVICES ALL WARRANTIES AND CONDITIONS, WHETHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, ANY (IF ANY) WARRANTIES OR CONDITIONS OF OR RELATED TO: TITLE, NON-INFRINGEMENT, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, LACK OF VIRUSES, ACCURACY OR COMPLETENESS OF RESPONSES, RESULTS, LACK OF NEGLIGENCE OR LACK OF WORKMANLIKE EFFORT, QUIET ENJOYMENT, QUIET POSSESSION, AND CORRESPONDENCE TO DESCRIPTION. THE ENTIRE RISK ARISING OUT OF USE OR PERFORMANCE OF THE SOFTWARE COMPONENT AND ANY SUPPORT SERVICES REMAINS WITH YOU.

16. "Software Crisis" Term coined by DoD years ago Problem Today: complexity of problems addressed by software has outpaced improvements in software creation process demand Programmers supply Time

17. "We have repeatedly reported on cost rising by millions of dollars, schedule delays, of not months but years, and multi-billion-dollar systems that don't perform as envisioned. … The understanding of software as a product and of software development as a process is not keeping pace with the growing complexity and software dependence of existing and emerging mission-critical systems." Government Accounting Office

18. "Few fields have so large a gap between best current practice and average current practice." Department of Defense

20. WhySoftwareQualityAssurance? • Softwarefailuresarecostly! • One hour downtime costs >$6M on average (Gartner, 1998) • Account for 30% of system failures (Marcus, 2000) • Contribute to 55% of most severe vulnerabilities (CERT) • Cost billions annually (NIST, 2002) • $22.2B annual potential cost reduction from feasible • infrastructureimprovements (NIST, 2002)

21. 2. What is Software Quality?

22. Software Quality

23. Software Quality Assurance (SQA)

24. The Objective of SQA • Assuring an acceptable level of confidence that the software will conform to functional technical requirements • Assuring an acceptable level of confidence that the software will conform to managerial scheduling and budgetary requirements • Initiation and management of activities for the improvement and greater efficiency of software development and SQA activities

25. Software Errors, Faults, Failures Software errors are sections of the code that are partially or totally incorrect as a result of a grammatical, logical or other mistake made by a systems analyst, a programmer, or another member of the software development team Software faults are software errors that cause the incorrect functioning of the software during a specific application Software faults become software failures only when they are “activated”, that is, when a user tries to apply the specific software section that is faulty. Thus, the root of any software failure is a software error

29. 9 Causes of Software Errors • Faulty requirements definition • Client–developer communication failures • Deliberate deviations from software requirements • Logical design errors • Coding errors • Non-compliance with documentation and coding instructions • Shortcomings of the testing process • Procedure errors • Documentation errors

30. Cost of Errors "Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product. … Although all errors cannot be removed, more than a third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. These are the savings associated with finding an increased percentage (but not 100 percent) of errors closer to the development stages in which they are introduced. Currently, over half of all errors are not found until "downstream" in the development process or during post-sale software use.“ (US Dept of Commerce, June 2002)

31. Some Famous Software Errors Therac-25 Patriot Missile System ESA's Ariane 5 Launch System Source: www.wikipedia.org

32. Therac-25 - the problem When operating in soft X-ray mode, the machine was designed to rotate three components into the path of the electron beam, in order to shape and moderate the power of the beam. … The accidents occurred when the high-energy electron-beam was activated without the target having been rotated into place; the machine's software did not detect that this had occurred, and did not therefore determine that the patient was receiving a potentially lethal dose of radiation, or prevent this from occurring.

33. Therac-25 - the reasons The design did not have any hardware interlocks to prevent the electron-beam from operating in its high-energy mode without the target in place The hardware provided no way for the software to verify that sensors were working correctly The equipment control task did not properly synchronize with the operator interface task, so that race conditions occurred if the operator changed the setup too quickly. This was evidently missed during testing, since it took some practice before operators were able to work quickly enough for the problem to occur The software set a flag variable by incrementing it. Occasionally an arithmetic overflow occurred, causing the software to bypass safety checks

34. Patriot Missile System On February 25, 1991, the Patriot missile battery at Dharan, Saudi Arabia had been in operation for 100 hours, by which time the system's internal clock had drifted by one third of a second. For a target moving as fast as an inbound TBM, this was equivalent to a position error of 600 meters The radar system had successfully detected the Scud and predicted where to look for it next, but because of the time error, looked in the wrong part of the sky and found no missile. With no missile, the initial detection was assumed to be a spurious track and the missile was removed from the system. No interception was attempted, and the missile impacted on a barracks killing 28 soldiers

35. Ariane 5 Rocket June 4, 1996 was the first test flight of the Ariane 5 launch system. The rocket tore itself apart 37 seconds after launch, making the fault one of the most expensive computer bugs in history. The Ariane 5 software reused the specifications from the Ariane 4, but the Ariane 5's flight path was considerably different and beyond the range for which the reused code had been designed. Specifically, the Ariane 5's greater acceleration caused the back-up and primary inertial guidance computers to crash, after which the launcher's nozzles were directed by spurious data. Pre-flight tests had never been performed on the re-alignment code under simulated Ariane 5 flight conditions, so the error was not discovered before launch. Because of the different flight path, a data conversion from a 64-bit floating point to 16-bit signed integer caused a hardware exception (more specifically, an arithmetic overflow, as the floating point number had a value too large to be represented by a 16-bit signed integer). Efficiency considerations had led to the disabling of the exception handler for this error. This led to a cascade of problems, culminating in destruction of the entire flight.

36. 3. Software Quality Factor

37. Question To know that quality has improved, it would be helpful to be able to measure quality. How can we measure quality?

38. Software Quality Measurements

39. Product Aspect

40. McCall’s Factor Model • McCall’s factor model classifies all software requirements into 11 software quality factors • The 11 factors are grouped into three categories – product operation, product revision and product transition: • Product operation factors: Correctness, Reliability, Efficiency, Integrity, Usability • Product revision factors: Maintainability, Flexibility, Testability • Product transition factors: Portability, Reusability, Interoperability

42. Alternative Factor Models • The two factor models from the late 1980s, alternatives to the McCall classic factor model: • The Evans and Marciniak factor model • The Deutsch and Willis factor model • These alternative models suggest adding five factors to McCall’s model. Two of these factors are very similar to two of McCall’s factors; only three factors are “new”: • Both models add the factor Verifiability • The Deutsch and Willis model adds the factors Safety and Manageability

44. ISO 9126 Software Quality Factors Functionality Reliability Usability Efficiency Maintainability Portability

45. BobotPengukuran Product • Dapatdipelihara (maintainable) • Dapatdiandalkan (reliable) • Aman (Secure) • EfektifdanEfisien • Kemampupakaian (Usabilitas) • Kemampupakaiankembali (Reusabilitas) - Software game harusinteraktifdanresponsif - Software phone switching harushandal - Software ecommerce danperbankanharusaman (secure)

46. FaktorKualitasPerangkatLunak (Taxonomy McCall)

47. ContohPengukuran Product* Fa = w1c1 + w2c2 + … + wncn F= Factor, W= Weight, C=Criteria *Source: MengukurKualitasPerangkatLunak, RomiSatriaWahono.Net

48. Process Aspect

49. Capability Maturity Model (CMM) • Level 1 – Initialtanpaprosedurdan planning, tidakkonsisten • Level 2 – Repeatableadamanajemen, jaminankualitas, prosedur, individual performance tanpa model formal • Level 3 – Definedprosesterdefinisi, danmengarahkeperbaikanprosessecarakualitatif • Level 4 – Managedperbaikandanprediksiprosessecarakuantitatif • Level 5 – Optimizingmemperbaikiprosessecaraberkesinambungan, inovatif, direncanakan, dianggarkandan integral dalamprosesorganisasi Capability Maturity Model (CMM), Software Engineering Institute

50. History • 1986 - Effort started by SEI and MITRE Corporation • assess capability of DoD contractors • First version published in 1991 • closely related to TQM • goal is customer satisfaction • not required that customer be "delighted"