16.355 Software Engineering Concepts 16.842 System Engineering

1. 16.355 Software Engineering Concepts16.842 System Engineering Prof. Nancy Leveson Fall 2013

2. Is There A Problem? • AAS (FAA Advanced Automation System) [$3-6B] • FBI CIC • IRS Modernization Program [$4B] • Denver Baggage System [$200M] • Mars Program losses (MCO, MPL, other close calls) • Hubble • X-33 Reusable Launch Vehicle • Military aircraft • Dreamliner (Boeing 787)

3. Software Difficulties • Head of AF Systems Command: “Software is the Achilles heel of weapons development” • 7 out of every 10 major weapons development programs are encountering software problems and the rate is increasing • Failure or cancellation rate of large software is over 20% • 65% of very large systems (over 1,000,000 LOC) are cancelled before completion • Average cancelled project in U.S. is about a year behind schedule and has consumed 200% of expected budget. • Of completed projects • 2/3 experience schedule delays and cost overruns (bad estimates?) • 2/3 experience low reliability and quality problems in first year of deployment

4. Have you ever been on a project where the software was never finished or used? • Have you been on a system engineering project that ran into serious difficulties?

5. Some Frequently Cited Factors • Underestimation of complexity • Failure to establish appropriate control over requirements and/or scope • Inadequate communication • Failure to engage stakeholders • Inadequate testing • Lack of oversight or poor project management • Poor quality implementations • Lack of risk management • Failure to specify/address performance requirements • Poorly planned/managed transitions • Excessive process to prevent previous problems

6. Types of Problem Projects (Yourdan) • Mission Impossible Likely to succeed, happy workers • Ugly Likely to succeed, unhappy workers • Kamikaze Unlikely to succeed, happy workers • Suicide Unlikely to succeed, unhappy workers

7. Software Death March Projects • Feature creep • Thrashing • Integration problems • Overwriting source code (configuration control issues) • Constant re-estimation • Redesign and rewriting during test • No documentation of design decisions • Etc.

8. Understanding the Problem

9. Understanding the Problem (2)

10. Laws of Maintenance (Belady and Lehman) • Software will continually change • Software will become increasingly unstructured as it changes

11. Why is software engineering hard? • Is software engineering more difficult than hardware engineering? • Why or why not?

12. The Computer Revolution

13. Accident with No Component Failures

14. Abstraction from Physical Design Software engineers are doing physical design Most operational software errors related to requirements (particularly incompleteness) Software “failure modes” are different Usually does exactly what you tell it to do Problems occur from operation, not lack of operation Usually doing exactly what software engineers wanted Autopilot Expert Requirements Design of Autopilot   Software Engineer 

15. The Curse of Flexibility • “Software is the resting place of afterthoughts” • No physical constraints • To enforce discipline on design, construction, and modification • To control complexity • So flexible that start working with it before fully understanding what need to do • The untrained can get partial success “Scaling up is hard to do” • “And they looked upon the software and saw that it was good. But they just had to add one other feature …”

16. Other Factors • Large discrete state spaces • Continuous vs. discrete math • Cannot test exhaustively • Intangibility • Invisible interfaces • Hard to experiment with and manage • Hard to diagnose problems • No historic usage information • To allow measurement, evaluation, and improvement of standard designs over time • Always specially constructed • Usually doing new things

17. Black Box Testing

18. White Box Testing

19. White Box Testing (2)

20. Class Objectives • Students will be able to evaluate SE techniques and approaches “It is important that students bring a certain ragamuffin barefoot irreverence to their studies. They are here not to worship what is known, but to question it.” Jacob Bronowski, The Ascent of Man “The developed theories … have rarely been subjected to empirical testing, and so their value remains unknown. They provide zealots with opportunities to market a rash of seminars and courses and to flood the literature with papers advocating the new technologies. When the theories are subjected to testing, what little evidence has been obtained sometimes suggests that the claimed benefits, in fact, may not exist.” Vessey and Weber Arguments may involve: Proof by vigorous handwaving Unsupported hypotheses False analogies • Students will be able to exercise professional judgement in selecting an approach for a particular project

21. Assignments • No programming or class projects • Reading summaries • Main ideas or themes • Critical evaluation or specific questions to answer • Any additional thoughts • Some additional short assignments • Additional reading for those preparing for Quals Exam