Software Engineering Issues in System Dependability

1. Software Engineering Issues in System Dependability Dr Martyn Thomas CBE FREng DIRECTOR, MARTYN THOMAS ASSOCIATES

2. 60 Years of Computing 40 Years of Software Engineering • The first stored program computer ran its first program in Manchester, UK in June 1948. • The first commercial software was the Bakeries Valuations for the Lyons Electronic Office (UK 1951) • The first software engineering conference was held in Garmisch, Germany 40 years ago in October 1968

3. Privacy, Security and Software Engineering • To process and manage confidential information, we need secure, dependable computer systems. • Creating secure and dependable systems cost-effectively needs strong software engineering. • Commercial software development is usually not good enough to create secure systems with very few errors. • A Study by the US National Security Agency found that methods based on mathematics can provide a cost-effective solution. • The Challenges of Complex IT Projects

4. By Secure SystemsI mean systems that provide the required service and do not permit unauthorised access or interference. • By Dependable SystemsI mean systems that can be trusted to operate as required, because there is strong evidence that they do not contain critical errors.

5. Most commercial software contains many errors • Researchers have reported fault densities of 1 to 30 defects per KLoC (thousand lines of source program) in commercial software in service. • The UK Ministry of Defence found over 20 defects per KLoC in the operational software in the C130J military transport aircraft (including many safety critical faults). • Sources: German & Mooney, Air Vehicle Static Code Analysis - Lessons Learnt. 2001. Pfleeger & Hatton,IEEE Computer, pp33-42, February 1997. Fenton & Ohlsson, Quantitative Analysis of Faults and Failures in a Complex Software System, http://www.md.kth.se/RTC/SC3S/papers/FentonOhlsson105280_final.pdf

6. Security errors have caused great damage • “Several individual viruses and worms have caused events where damage was assessed at over $1billion each” Source: Software for Dependable Systems - Sufficient Evidence? National Academies Press, 2007 • Many security vulnerabilities arise from simple programming errors, such as “buffer overflows” where the length of input into an array is not checked and this allows a malicious user to overwrite the program.

7. Testing alone can never be enough to show that a program is dependable • To test all possible paths with all possible data is completely impractical. • Finding bugs and fixing them cannot lead to secure, dependable software. • Critical software errors may remain hidden for decades.

8. Testing software tells you that the tests work – not that the software works Continuous behaviour means you can interpolate between test results Discrete behaviour means that you can’t!

9. An Experiment by the NSA • A recently published experiment by the US National Security Agency shows that the use of mathematically formal methods can produce dependable software, cost effectively.

10. TOKENEER ID Station Source: Slides by Randolph Johnson, NSA

11. The Tokeneer experiment • Praxis developed software for the Tokeneer ID Station using their Correct by Construction (C by C) methods • NSA asked an independent team to find errors • Praxis then trained a group of NSA staff in C by C • The NSA staff modified the TIS to add new functions • All the specifications, designs, tests, analyses, proofs and other documentation were collected • Productivity was analysed and compared with previous projects

12. Correct by Construction Software Engineering source: Peter Amey, Praxis HIS • A combination of formal methods and agile development • Precise notationsand incremental development • Three basic principles: • Make it hard to introduce errors (Z, SPARK) • Find any errors almost immediately (SPARK Examiner) • Generate evidence of dependability throughout development, as a natural by-product of the process

13. C by C Development Source: Peter Amey, Praxis HIS

14. C by C Guiding Principles • Write right • Step, don’t leap • Say something once • Check here before going there • Explain your decisions • Use the right tools for the job Source: Peter Amey, Praxis HIS

15. Write Right • Capture information, requirements, designs and code unambiguously and accurately. • Use notations with strong semantics

16. Step, Don’t Leap • Make each development step semantically small, to reduce errors and improve checking.

17. Say something once • Make each document have a clear purpose and write information in only one place. This aids clarity and avoids inconsistency.

18. Check here before going there • Verify each step before moving on. Check for consistency against something else – usually the previous design step.

19. Explain your decisions • Document design decisions and explain why they are justified – that will help you be sure they are right, and help you and others to understand them later.

20. Use the right tools • Use the appropriate verification method for each property, whether tool-supported proof, informal peer review, static analysis or testing.

21. Tokeneer was written in SPARK • SPARK enforces type security and enables deep static analysis: • no buffer overflows • complete elimination of several classes of errors • complete analysis of program flow and data flow • SPARK formal annotations make it possible to prove that the software implements the specification http://www.praxis-his.com/sparkada/

22. Statistics of Tokeneer System Source: R Johnson, NSA

23. Statistics of Tokeneer System Source: R Johnson, NSA

24. C by C is a practical approach • The NSA found that the Correct by Construction methods could be successfully taught to postgraduate engineers in a few weeks. • Summer intern students successfully added new features to the Tokeneer ID Station using C by C, with high productivity and zero defects

25. All the Tokeneer materials have been published • http://www.adacore.com/tokeneer • The materials allow the entire project to be repeated, including all the static analysis steps and all the proofs. • The NSA has licensed the materials under the GPL (GNU General Purpose Licence).

26. IT projectsoften overrun

27. IT projectsoften overrun • Standish “Chaos Chronicles” (2004 edition): • 18% of projects “failed”; (cancelled before completion) • 53% of projects “challenged” (operational, but seriously over budget and/or over time with fewer features or functions than initially specified) • Typical Standish figures: • Cost overruns on 43% of projects; and • Time overruns on 82% of projects. • Oxford University/Computer Weekly 2003 study: • 10% of UK projects “failed”; and • 75% of UK projects “challenged”.

28. Getting the Requirements right • In the UK, many Government IT projects have failed. • Our software industry is very optimistic; so are our politicians. • This can lead to “irrational exuberance” (Alan Greenspan’s phrase) • There is no such thing as an IT Project - only Business Change enabled by IT. The IT costs are typically < 50% • The Royal Academy published a study in 2004: The Challenges of Complex IT Projects which recommended the use of systems architects

29. How do you get the right technical solution to a business requirement? • USE AN ARCHITECT! A specialist in requirements, high-level designand working with engineers http://www.raeng.org.uk/news/publications/list/reports/Complex_IT_Projects.pdf

30. Summary • Software that needs to be secure must be dependable • Dependability can only be achieved with the use of mathematically formal methods • Formal methods are now practical and cost-effective for use in commercial and industrial software development • The Tokeneer experiment allows software developers to benchmark their own methods against Correct by Construction and to experiment with developing near zero-defect software, at no increase in cost. • Our engineering Academies need to promote the use of scientifically valid methods and tools, to reduce the number of project failures and security disasters.