Balancing Agility and Discipline: Evaluating and Integrating Agile and Plan-Driven Methods

1. Barry BoehmUniversity of Southern CaliforniaRichard TurnerOUSD(AT&L)/DS/SE (George Washington University) Balancing Agility and Discipline:Evaluating and Integrating Agile and Plan-Driven Methods Based on Balancing Agility and Discipline: A Guide for the Perplexed,B. Boehm and R. Turner, Addison Wesley, 2004

2. Outline • Tutorial learning objectives • Survey of assumptions about participants • General software trends and implications • Sources of perplexity about agile, plan-driven methods • Overview of agile and plan-driven methods • XP, TSP “days in the life” • Comparisons of differences, strengths, weaknesses • Risk-based balance of agility and discipline • Small, medium, large examples • Observations and Way Forward • Hands-on exercise • Conclusions; review of learning objectives

3. Tutorial Learning Objectives • Practitioners and managers • Understand agile, plan-driven method strengths, difficulties • Learn risk-based approach to tailor hybrid methods • Practice in formulating organizational strategy • Researchers • Understand research issues, opportunities • Techniques for achieving rapid quality • Empirical analyses • Educators • Understand differences in agile, plan-driven approaches • Understand teaching challenges, opportunities • Survey courses, project courses

4. Assumptions About Participants - Surveyed at Tutorial • Generally familiar with plan-driven methods • Waterfall, incremental, spiral, RUP, PSP/TSP • Software CMM, CMMI, SPICE, ISO I2207 • Less familiar with agile methods • XP, Scrum, Crystal, DSDM, FDD • Representative of mixed domains • Large/small projects, research, education • Variety of software applications domains • Increasing need for high software dependability • Increasing software complexity, speed of change

5. Traditional Development Standalone systems Stable requirements Rqts. determine capabilities Control over evolution Enough time to keep stable Stable jobs Failures noncritical Reductionist systems Repeatability-oriented process, maturity models Information Technology Trends • Current/Future Trends • Everything connected (maybe) • Rapid requirements change • COTS capabilities determine rqts. • No control over COTS evolution • Ever-decreasing cycle times • Outsourced jobs • Failures critical • Complex, adaptive, emergent systems of systems • Adaptive process models

6. Background • Two approaches to software development • Disciplined (SW-CMM, document-based, heavy process) • Agile (XP, tacit knowledge, light process) • Both have strengths and weaknesses • Agile and disciplined proponents are believers • Many of us are perplexed

7. Key Definitions • Agile method – • one which fully adopts the four value propositions and twelve principles in the Agile Manifesto. • Discipline – (per Webster) • control gained by enforcing obedience or order; • orderly or prescribed conduct or pattern of behavior; • self-control. • Plan-driven – • a description for disciplined methods (order is often defined in plans) • Plan – (per Webster) • a method for achieving an end (a process plan); • an orderly arrangement of parts of an overall design (a product plan). • We assume that such plans are documented.

8. The Agile Manifesto • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: That is, while there is value in the items on the right, we value the items on the left more.

9. Plan-Driven Methods Are Evolving Too • Evolutionary acquisition and spiral development • U.S. DoD Directive 5000.1, Instruction 5000.2 • Risk-driven level of detail • Of processes: peer reviews vs. Critical Design Reviews • Of products: GUI prototypes vs. specifications • Of heavyweight methods: Team Software Process, Rational Unified Process

10. Sources of Perplexity • Distinguishing method use from method misuse • Citing method misuse vs. intended use • Claiming XP use when simply “not documenting” • “CMM Level 4 Memorial Library” of 99 2-inch binders • Overgeneralization based on the most visible instances • XP is Agile; CMM is Plan-Driven • Multiple definitions • Quality: customer satisfaction or compliance? • Claims of universality • The pace of IT change is accelerating and Agile methods adapt to change better than disciplined methods therefore Agile methods will take over the IT world. • Software development is uncertain and the SW-CMM improves predictability therefore all software developers should use the SW-CMM

11. Sources of Perplexity (2) • Early success stories • Chrysler project that successfully birthed XP was later cancelled • Cleanroom has never made it into the mainstream • Purist interpretations (and internal disagreements) • “Don't start by incrementally adopting parts of XP. Its pieces fit together like a fine Swiss watch“ • "An advantage of agile methods is that you can apply them selectively and generatively." • "If you aren't 100% compliant with SW CMM Level 3, don't bother to bid" • "With the CMMI continuous interpretation, you can improve your processes in any order you feel is best."

12. Outline • Tutorial learning objectives • Survey of assumptions about participants • General software trends and implications • Sources of perplexity about agile, plan-driven methods • Overview of agile and plan-driven methods • XP, TSP “days in the life” • Comparisons of differences, strengths, weaknesses • Risk-based balance of agility and discipline • Small, medium, large examples • Observations and Way Forward • Hands-on exercise • Conclusions; review of learning objectives

13. Various Agile Methods Available • Adaptive Software Development (ASD) • Agile Modeling • Crystal methods • Dynamic System Development Methodology (DSDM) * eXtreme Programming (XP) • Feature Driven Development • Lean Development • Scrum

14. XP Principles – I • Philosophy: Take known good practices and push them to extremes • “If code reviews are good, we’ll review code all the time” • “If testing is good, we’ll test all the time” • “If design is good, we’ll make it part of everybody’s daily business”

15. XP Principles – II • “If simplicity is good, we’ll always leave the system with the simplest design that supports its current functionality” • “If architecture is important, everybody will work defining and refining the architecture all the time” • “If integration testing is important, then we’ll integrate and test several times a day”

16. XP Principles – III • “If short iterations are good, we’ll make the iterations really, really short – seconds and minutes and hours, not weeks and months and years” • “If customer involvement is good, we’ll make them full-time participants”

17. The Planning Game Small Releases Metaphor Simple Design Testing Refactoring Pair Programming Collective Ownership Continuous Integration 40-hour Week On-site Customer Coding Standards XP: The 12 Practices-Kent Beck, Extreme Programming Explained, Addison Wesley, 1999 -Used generatively, not imperatively

18. XP Characteristics

19. The Team Software Process (TSP)-Watts Humphrey, Introduction to the ISP, Addison Wesley, 2000 • Scale-up of Personal Software Process (PSP) • To about 20-person teams • Strong emphasis on planning and control • 21 scripts, 5 roles/activities, 21 forms • Risk-driven tailoring can make process more agile

20. Example TSP Role Script

21. TSP Characteristics

22. Days In The Life: XP and TSP • Comparison of two development teams • Product: • Tool to process complex sales report generated by legacy system • Prototype delivered • Task is to enhance and port prototype • 8 month projected schedule • Estimated size: 20,000 SLOC

23. TSP/PSP Training Each member has 125-hour PSP for Engineers 2 Members 5-day launch coach training Tools and environment Web-based TSP data collection tool OO tools Individual offices and conference room Project planning 4-day launch session Develop/tailored all processes 180 tasks identified and plan accepted by stakeholders XP Training 2 members attended 40-hour XP workshop In-house presentations and mentoring Tools and environment Automated test suite development tool OO tools Bullpen and Conference room/lounge Project planning 1-day customer exploration 2-day follow-up to prioritize Agreed on 2-week iteration length, 3-iteration release length, and release schedule Background Information

24. Normal and Crisis Days • Used Friday as common day • Crisis days are plan-breakers • New requirements • Short schedule • Please read the handouts and we’ll discuss

25. Comparison • Differences • Depth and scope of metrics • Specificity of process • Level and scope of reporting • Customer interface • Similarities • Collaborative teams • Well-defined roles • Spiral, risk- and increment-driven process • Measurement • Test first (test-driven) approach • Bottom Line • Both were able to confidently push-back when necessary

26. Outline • Tutorial learning objectives • Survey of assumptions about participants • General software trends and implications • Sources of perplexity about agile, plan-driven methods • Overview of agile and plan-driven methods • XP, TSP “days in the life” • Comparisons of differences, strengths, weaknesses • Risk-based balance of agility and discipline • Small, medium, large examples • Observations and Way Forward • Hands-on exercise • Conclusions; review of learning objectives

27. Finding Middle Ground • A pragmatic view • Both approaches have “home grounds” • A broad range of environments and needs • Trends require better handling of rapid change • Processes should be the right weight for the job • We believe risk-based analysis is the key to finding middle ground

28. Contrasts and Home Grounds • Comparison is difficult, but we found 4 areas where there are clear differences • Application • Project goals, size, environment; velocity • Management • Customer relations, planning and control, communications • Technical • How the software is developed • Personnel • The type and competency of developers and stakeholders

29. Application Characteristics • Primary goals • A: Rapid value, responsiveness to change • P: Predictability, stability, high assurance • Size • A: Small to medium • P: Large • Environment • A: Turbulent, high change • P: Stable, requirements defined • Project velocity • A: Code, learn, and fix • P: Architect for parallel, incremental development

30. Management Characteristics • Customer Relations • A: Dedicated, collocated; where feasible • P: Contractual; increasingly evolutionary • A: Trust through working software and participation • P: Trust through process maturity evaluations • Planning and Control • A: Means to an end • P: Anchor processes, communication • Project Communications • A: Tacit, interpersonal knowledge • P: Explicit, documented knowledge

31. Technical Characteristics • Requirements • A: Adjustable, informal stories • P: Formally baselined, complete, consistent specifications • Development • A: Simple Design • P: Architecture-based design • Testing • A: Automated, test-driven • P: Planned, requirements-driven

32. Technical Characteristics (2)Cost of Change: Beck, Li Beck Li Li

33. Changes in High-stakes Constraints Cost of Change Most Changes Time Technical Characteristics (3) Cost of change: Poppendieck • At least two cost-escalation curves • Lean development: Architect to defer high-stakes decisions – e.g. COTS • Easier to change an unmade decision

34. Technical Characteristics (4)Cost of Change: TRW Beck Li

35. Technical Characteristics (5)How Much Architecting? Beck 10,000 KSLOC Sweet spot 100 KSLOC Liu 10 KSLOC

36. Personnel Characteristics • Customers • A: CRACK customers throughout development • P: CRACK customers early • CRACK: Collaborative, Representative, Authorized, Committed, and Knowledgeable • Developers • A: Heavy mix of high caliber throughout • P: Heavy mix early with lower capability later • Culture • A: Many degrees of freedom (Thrives on chaos) • P: Clear policies and procedures (Thrives on order) • Collocated customers are difficult to find/keep/keep current on either side

37. Personnel Characteristics (2)Modified Cockburn Levels Level Characteristics 3 Able to revise a method (break its rules) to fit an unprecedented new situation 2 Able to tailor a method to fit a precedented new situation 1A With training, able to perform discretionary method steps (e.g., sizing stories to fit increments, composing patterns, compound refactoring, complex COTS integration). With experience can become Level 2. 1B With training, able to perform procedural method steps (e.g. coding a simple method, simple refactoring, following coding standards and CM procedures, running tests). With experience can master some Level 1A skills. -1 May have technical skills, but unable or unwilling to collaborate or follow shared methods.

38. Summary of Home Grounds

39. Two Case Studies • Show how that agile and plan-driven methods can be extended • Provide examples of success and lessons learned • ThoughtWorks Lease Management • Agile extended with plan-driven • CCPDS-R • Plan-driven overlaid with agile concepts

40. Thoughtworks Lease Management • XP replaced ineffective traditional development • Problems when project moved beyond XP assumptions • The effort to develop or modify a story really does not increase with time and story number • Trusting people to get everything done on time is compatible with fixed schedules and diseconomies of scale • Simple design and YAGNI scale up easily to large projects • Disciplined practices enabled XP to scale up • High-level architectural plans to provide essential big-picture information • More careful definition of milestone completion criteria to avoid “finishing” but not finishing • Use of design patterns and architectural solutions rather than simple design to handle foreseeable change

41. CCPDS-R • USAF Command Center Processing and Display System Replacement for early missile warning • Government contract environment required plan-driven approach, project needed agility • Practices implemented to provide agility mapped well to the Agile Manifesto • Individuals and Interactions over Processes and Tools • Milestone content was redefined • Architecture was organized around the performers’ skill levels • Working Software over Comprehensive Documentation • Later PDR demonstrated working software for high-risk areas • Customer Collaboration Over Contract Negotiation • Used COCOMO for cost and schedule negotiations • Responding to Change Over Following a Plan • Automated plans • Architecture for re-use saved effort

42. CCPDS-R (2) Cost of Change Beck

43. Five Critical Decision Factors • Size, Criticality, Dynamism, Personnel, Culture

44. Comparing the Case Study Projects

45. Misconceptions

46. Outline • Tutorial learning objectives • Survey of assumptions about participants • General software trends and implications • Sources of perplexity about agile, plan-driven methods • Overview of agile and plan-driven methods • XP, TSP “days in the life” • Comparisons of differences, strengths, weaknesses • Risk-based balance of agility and discipline • Small, medium, large examples • Observations and Way Forward • Hands-on exercise • Conclusions; review of learning objectives

47. Using Risk to Balance Discipline and Agility - Overview

48. Risks Used in Risk Comparison • Environmental risks • Technology uncertainties • Many stakeholders • Complex system-of-systems • Legacy compatibility • Agility risks • Scalability • Use of simple design • Personnel turnover • Too-frequent releases • Not enough agile-capable people • Plan-driven risks • Rapid change • Need for rapid results • Emergent requirements • Not enough discipline-capable people

49. Three Examples • Agent-based systems • Small – Event Managers application • Medium – SupplyChain.com application • Large – National Information System for Crisis Management (NISCM) application • Each example results in a development strategy based on risk analyses

50. Event Managers Project • Small startup company • Diverse set of smaller agent-based planning applications • This project: support for managing the infrastructure and operations of conferences and conventions • Widening variety of options and interdependencies, makes an agent-based approach highly attractive