No Silver Bullet Essence and Accidents of Software Engineering Frederick P. Brooks, Jr.

1. No Silver BulletEssence and Accidents of Software EngineeringFrederick P. Brooks, Jr. Prepared by Jinzhong Niu August 20, 2014

2. Frederick P. Brooks, Jr. • Kenan Professor of CS at Univ. of North Carolina - Chapel Hill • Achievements • IBM OS/360 • “The mythical Man-Month” • Honors and Awards • A.M. Turing Award, ACM (1999) • National Medal of Technology (1985) Readings in Computer Science: Software Engineering

3. About this paper • Proc. IFIP Congress 1986, Information Processing 86 • IEEE Computer, Vol. 20, No. 4, Apr. 1987 • The Mythical Man-Month, 2nd Edition, 1995 • Software Engineering, edited by Merlin Dorfman and Richard Thayer, Wiley-IEEE Press • 1st Edition, 1996 • 2nd Edition, 2002 Readings in Computer Science: Software Engineering

4. What are werewolves and silver bullets? • Werewolf • one of the oldest monster legends • popular movie topic • Silver bullet • the only thing that can kill werewolves Even a man who is pure at heart,And says his prayers at night,Can become a wolf when the wolfbane blooms,And the moon is full and bright. -- From “The Wolf Man” Readings in Computer Science: Software Engineering

5. “Essence” and “Accident” • Essence • noun, the permanent as contrasted with the accidental element of being • the mental crafting of the conceptual constructs • Accident • noun, a nonessential property or quality of an entity or circumstance; appurtenant(rather than misfortune or occurring by chance) • the implementation process of conceptual constructs Readings in Computer Science: Software Engineering

6. Overview • Software development is werewolf, but there is no silver bullet because of its essential difficulties. • What is the nature of software development? (Why does it have to be hard?) • Did past breakthroughs solve the problem? • Is there any potential solution nowadays? • Will the problem be attacked in the future? Readings in Computer Science: Software Engineering

7. Problem solving strategy 知己知彼，百战不殆。 —— 《孙子兵法》 Know your enemy and know yourself; in a hundred battles, you will never be defeated. ---SUN TZU ON THE ART OF WAR Readings in Computer Science: Software Engineering

8. Why does software engineering have to be hard? • Outside • Computer hardware progress is an exception. • Inside • There are essential difficulties which are hard to be attacked. Readings in Computer Science: Software Engineering

9. Essential difficulties • Complexity • Conformity • Changeability • Invisibility Readings in Computer Science: Software Engineering

10. Essential difficulties: Complexity • A system is usually defined as a collection of components, which interact with one another. • Software is much more complex than any other human construct. • The number of elements • The interaction between elements Readings in Computer Science: Software Engineering

11. Essential difficulties: Complexity --- Cont. • A variety of problems are caused. • Technical • Decrease of reliability, usability, extensibility, safety • Managerial • Difficulty of communication between team members • Difficulty of keeping a clear integrated overview and all the loose ends • Difficulty of personnel turnover due to tremendous learning and understanding burden Readings in Computer Science: Software Engineering

12. Why high complexity? • Software varies. • A colorful world needs colorful software systems, because “software has become the dominant technology in many if not most technical systems. It often provides the cohesiveness and data control that enable a complex system to solve problems.” [SwSE, Richard Thayer] • A single piece of software involves high complexity. • High conformity Readings in Computer Science: Software Engineering

13. Essential difficulties: Conformity • Unlike physics where a terrible but invariable complexity exists, software has to conform many human institutions and system interfaces, the number of which is still swelling all the time. • Redesign of the software alone cannot simplify out the complexity. Readings in Computer Science: Software Engineering

14. Essential difficulties: Changeability • Software is constantly subject to pressures for change. • Successful software DOES change frequently. Readings in Computer Science: Software Engineering

15. Essential difficulties: Changeability --- Cont. • Why? • Necessity:Software embodies function, which most feels the pressures of change in a system. • Successful software is hoped to function over time. • It is hoped to function in new domains. • Feasibility:Software, pure thought-stuff, is infinitely malleable. Readings in Computer Science: Software Engineering

16. Essential difficulties: Invisibility • Software is invisible in the sense that it is not inherently embedded in space. • Software structure is difficult to visualize in a hierarchical fashion. Readings in Computer Science: Software Engineering

17. Did past breakthroughs solve the problem? • No. What they attacked are accidental difficulties not essence. • Give me some examples! • High-level languages • Time-sharing • Unified programming environments Readings in Computer Science: Software Engineering

18. High-level languages • The development of high-level languages is credited with • at least a factor of five in productivity, • concomitant gains in reliability, simplicity, and comprehensibility. • It, however, eliminates only the complexity related to lower level constructs that are not inherent in software. • The level of our thinking about data structures, data types, and operations is steadily rising, but at an ever decreasing rate, and approaches closer and closer to the sophistication of users. Readings in Computer Science: Software Engineering

19. High-level languages --- Cont. Readings in Computer Science: Software Engineering

20. Time-sharing • Time-sharing eliminates the slow turnaround of batch programming, and keeps fresh in mind the grasp of a complex system. • The benefit of time-sharing is to be boundary due to the human threshold of noticeability. Readings in Computer Science: Software Engineering

21. Unified programming environments • Unified programming environments enable related individual tools to work together in an automatic manner. They thus free programmers from the burden of various manual operations. • By its very nature, the fruit is and will be marginal. Readings in Computer Science: Software Engineering

22. Isthere any potential solution nowadays? • Ada and other high-level language advances ? • Object-oriented programming ? • Artificial Intelligence ? • Expert Systems ? • “Automatic” programming ? • Graphical programming ? • Program verification ? • Environments and tools ? • Workstations ? Readings in Computer Science: Software Engineering

23. Ada • Ada, one of the most touted recent development, not only reflects evolutionary improvements in language concepts, but indeed embodies features to encourage modern design and modularization. • Nevertheless, it is just another high-level language and will not prove to be the silver bullet. Readings in Computer Science: Software Engineering

24. Object-oriented programming • Two orthogonal concepts representing real advances: • abstract data types • hierarchical types • OO Concepts: • encapsulation • abstraction • inheritance • polymorphism • dynamic binding Readings in Computer Science: Software Engineering

25. Object-oriented programming --- Cont. • OO represents real advances in the art of building software. • Nevertheless, they remove only accidental difficulties from the expression of the design, rather than the design itself. Readings in Computer Science: Software Engineering

26. Artificial Intelligence • Terminological chaos – Two definitions: • AI-1: The use of computers to solve problems that previously could only be solved by applying human intelligence. • AI-2: The use of a specific set of programming techniques known as heuristic or rule-based programming. (expert system) Readings in Computer Science: Software Engineering

27. Artificial Intelligence --- Cont. • AI advancements facilitate HCI (Human Computer Interface). • However, the hard thing about building software is deciding what to say, not how to express. Readings in Computer Science: Software Engineering

28. Expert Systems • Definition: • a program containing a generalized inference engine and a rule base, takes input data and assumptions, explores the inferences derivable from the rule base, yields conclusions and advice, and explains its results by retracting its reasoning for the user • Advantages: • Inference-engine technology is application-independent. • The application-peculiar materials are encoded in the rule base in a uniform fashion, which regularizes the complexity of the application itself. Readings in Computer Science: Software Engineering

29. UserInterface Inputdata KnowledgeBase (Rules, Facts) InferenceEngine User advices Expert Systems --- Cont. Readings in Computer Science: Software Engineering

30. Expert Systems --- Cont. • Possible benefits: • Expert systems in software engineering field • Building software in the way expert systems work • Difficulties • How to generate automatically the diagnostic rules from program-structure specification • How to extract expertise and distill it into rule bases Readings in Computer Science: Software Engineering

31. “Automatic” programming • Automatic programming is actually a euphemism for programming with a higher-level language so that a solution could be given more easily. • There are some exceptions which have favorable properties: • Relatively few parameters are involved. • Many solutions are available. • Explicit rules are known to select solutions. • It is hard to generalize such special cases for the ordinary software systems. Readings in Computer Science: Software Engineering

32. Graphical programming • Computer graphics, which has been applied successfully in other fields, seems to be able to play a role in software design. • Nothing convincing has ever emerged from this approach. • The flowchart, considered as the ideal program-design medium, is a very poor abstraction of software structure. • The screens of today are too small to show detailed software diagrams. • In its nature, software is very difficult to visualize. Readings in Computer Science: Software Engineering

33. Program verification • Program verification seems promising to avoid immense effort upon implementation and testing by eliminating errors in the design phase. • No magic! • Verifications are so much work that only a few programs have been verified. • Verification cannot eliminate errors totally since mathematical proofs can also be faulty. • Specification, the baseline of verification, is usually incomplete and inconsistent. Readings in Computer Science: Software Engineering

34. Environments and tools • We have discussed this issue. Readings in Computer Science: Software Engineering

35. Workstations • More powerful computers surely facilitate software development. • But nowadays time of thinking, instead of waiting for computers' response, is the dominant activity of programmers.Magical enhancement thus cannot be expected. Readings in Computer Science: Software Engineering

36. Will the problem be attacked in the future? • The conceptual components of the task are now taking most of time. • We must consider those attacks that address the essence of the software problem. Readings in Computer Science: Software Engineering

37. Will the problem be attacked in the future? --- Cont. • Well, there may be some copper bullets: • Buy versus build • Requirements refinement and rapid prototyping • Incremental development • Greater designers Readings in Computer Science: Software Engineering

38. Buy versus build • It is common practice to buy off-the-shelf products nowadays due to the following reasons: • PC revolution has created many mass markets for software, which, together with zero replication cost of software, stirred the motivation for software companies to produce more and better software products. • Applicability of software is enhanced with the generalization of software tools and the constantly decreasing hardware/software cost ratio. Readings in Computer Science: Software Engineering

39. Requirements refinement and rapid prototyping • It is hardest to decide detailed technical requirements.Unfortunately even the clients themselves do not exactly know what they want. • So iterative extraction and refinement of product requirements are necessary. Readings in Computer Science: Software Engineering

40. Requirements refinement and rapid prototyping --- Cont. • A client cannot specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product. • Rapid prototyping may give clients a first-hand feel of what the product will be and a check for consistency and usability. Readings in Computer Science: Software Engineering

41. Incremental development • To develop software that has a comparable complexity as human brain, a similar process should be followed, i.e. incremental development. • Advantages: • The approach necessitates top-down design, thus allowing easy backtracking and detecting fundamental defects as early as possible. • An always working system stirs enthusiasm. Readings in Computer Science: Software Engineering

42. Greater designers • People is the key factor of solving problems. • Steps have been taken to raise the level of our practice from poor to good. • Curricula • Literature • Research organizations Readings in Computer Science: Software Engineering

43. Greater designers --- Cont. • The proposed next step is to develop ways to grow great designers. • Why? • Creative minds present state-of-the-art works, the benefits of which are order-of-magnitude compared with the average practices. • How? • Identify top designers as early as possible • Assign a career mentor to be responsible for the development of the prospect • Work out a career-development plan for each prospect • Provide opportunities for designers to interact with and stimulate each other Readings in Computer Science: Software Engineering

44. Readings in Computer Science: Software Engineering

45. Bullets towards NSB • We cannot abstract away the complexity without abstracting away the essence? • “Divide and conquer” strategy • Is a hierarchical model of software possible?We cannot always visualize software in hierarchical graphs? • The benefit of time-sharing is boundary? • The hardest single part of building a software system is deciding precisely what to build? • How about design? Readings in Computer Science: Software Engineering