Agile system development methods

1. Agile system development methods Henning Müller

2. Overview • Introduction • Motivations • Some vocabulary • Agile methods in general • Response to heavyweight waterfall model • Many small concepts • Stories of Agile projects

3. Henning Müller • Diploma in Medical Informatics • University of Heidelberg (1992-1997) • Daimler Benz research and technology • Portland, OR, USA (1997-1998) • PhD in image analysis • University of Geneva (1998-2002) • Medical image analysis and information systems • University and Hospitals of Geneva (2002-now) • HES SO (2007-now)

4. You • Background • Practical experiences (also concerning agile methods) • Expectations from this course • Brainstorming: What is Agile for you?

5. Goals • Obtain a solid knowledge on modern information system development methods • Agile vs. iterative methods vs. waterfall • Why Agile can help • Understand how to choose the right methodology for each project • Or components of it • There is no one size that fits all

6. Textbooks Ken SCHWABER, Mike BEEDLE,Agile Software development with SCRUM, Prentice Hall, 2002. Craig LARMAN,Agile and iterative development, Addison-Wesley, 2006.

7. Definition Agile methods (wikipedia) Agile software development is a conceptual framework for software engineering that promotes development iterations throughout the life-cycle of the project. There are many agile development methods; most minimize risk by developing software in short amounts of time. Software developed during one unit of time is referred to as an iteration, which may last from one to four weeks. Each iteration is an entire software project: including planning, requirements analysis, design, coding, testing, and documentation. An iteration may not add enough functionality to warrant releasing the product to market but the goal is to have an available release (without bugs) at the end of each iteration. At the end of each iteration, the team re-evaluates project priorities. Agile methods emphasize face-to-face communication over written documents. Most agile teams are located in a single open office sometimes referred to as a bullpen. At a minimum, this includes programmers and their "customers" (customers define the product; they may be product managers, business analysts, or the clients). The office may include testers, interaction designers, technical writers, and managers.

8. Definition iterative/incremental methods Incremental development is a scheduling and staging strategy in which the various parts of the system are developed at different times or rates, and integrated as they are completed. It does not imply, require nor preclude iterative development or waterfall development - both of those are rework strategies. The alternative to incremental development is to develop the entire system with a "big bang" integration. Iterative development is a rework scheduling strategy in which time is set aside to revise and improve parts of the system. It does not presuppose incremental development, but works very well with it. A typical difference is that the output from an increment is not necessarily subject to further refinement, and its' testing or user feedback is not used as input for revising the plans or specifications of the successive increments. On the contrary, the output from an iteration is examined for modification, and especially for revising the targets of the successive iterations.

9. Definition waterfall model Strict following of phases No way back Was often mandatory DARPA, CIA, governments Static No changes foreseen Not realistic for most software projects

10. Critics towards the waterfall model Software projects are not as static as many other product developments (scope creep) Up front specifications may not be realistic The final product is often not known at the start Risk in the waterfall model is pushed towards the end Plus “big bang” is risky per se A single contact with the customer is not enough

11. General problems with software development: Exercise Read the text “Standish Group Report 1995”, 1-4 What are the most surprising results? Read pages 5-7 Which factors are linked with software success? Why are so many software projects failing?

12. Standish Report 2000/2003 Read the one page abstract What has improved? What has not? Any propositions or explications?

13. Exercise: Read the text on agile methods and the lack of their use. Why is the waterfall model still used so broadly? What needs to be done to increase the number of agile development projects?

14. Product development • Software vs. other products • Cell phones • Plastic dinosaurs • Houses – architecture • Predictable vs. managing change • Estimate efforts and costs • Possible or not? • Plan a schedule, order all activities, details

15. Predictable/new product manufacturing • Most software is not a predictable or mass manufacturing problem. Software development is new product development • Often new, buggy technologies are used, increasing the novelty and unpredictability of processes • How long will this version of Java be supported?

16. Prevention of detailed up front specifications • Clients are not sure what exactly they want • Clients have difficulty stating everything • Details will only be revealed during the development • Details are very complex for people • External forces (actions of competitors, economic crisis) lead to change • Requirements to act quickly in Internet economy

17. Iterative development • Several iterations in sequences constitute the full development cycle • Every iteration is an independent sub (mini) project • Each iteration has analysis, design, development, testing • Goal is an iteration release • Incomplete but working prototype • Software across teams is integrated every release (subset of full system, no throw away)

18. Overview of iterative development

19. Overview of the project cycle

20. Length of iteration • Depends on methods • Sometimes fixed (SCRUM), sometimes variable • Most often recommended is 1-6 weeks • Depends on total length of the project • Cutting a project into smaller blocks helps to improve the success rate • Reduce complexity • Satisfaction in finishing off something

21. Project success vs. length of project

22. Risk-driven development • Reduce overall project risk early in the project • Start each iteration with the riskiest parts • If there is a real problem it is discovered quickly and can be addressed • Risk is not always easy to define or estimate • Probability and impact can be estimated

23. Risk in the waterfall model

24. Risk in iterative models

25. Client-driven development • Choice of features for next iteration comes from the client directly • Client steers the development iteration by iteration • Client has ongoing control of the project and makes most important decisions based on latest data, and not speculatively • Client is working directly with the developers

26. Timeboxing • Fix end date for each iteration without change • If too many features for completion, reduce the scope (improve estimates over time) • Remove features with lower priority • Always a running and tested system at the end of each iteration • Entire project can be timeboxed, or only iterations • Iterations can have varying length

27. Evolutionary and adaptive development • Requirements, plans and solutions evolve and are refined over time • No fully frozen specification at the start • Unpredictable discoveries can occur • System can be adapted based on feedback from users, clients, or developers • Internet economy is based on quick reaction times and quick changes based on the market

28. Evolutionary requirements • This does not mean that everything changes all the time! • No sloppy specifications • First iterations have often a larger percentage of requirements analysis • Most requirements are defined early on and iterations have increasingly short analyses • Requirements workshops in some early iterations for larger projects • Details on risky developments

29. Cone of uncertainty

30. Adaptive planning • Concentrate on high level influential parts • Architecturally important • High risk (load, usability, …) • Initial phase of high uncertainty • Make a more detailed planning 10%-20% into the project • After first developments and experiences • Initial exploratory phase

31. Cowboy coding • Absence of a defined method • Members of the team do what they think is right • No details on communication • This is not agile development! • Agile development follows methods and protocols, sometimes very strictly

32. Prizing • Short fixed-prize exploratory iterations are possible • Then larger-scale parts with possibility to stop at any moment • Software should be produced not documents • Waterfall method is easier to pay for • But is this realistic (promises) • Risk is mainly with the customer

33. Incremental/iterative delivery • Running product is delivered in several iterations • Delivery cycle is usually different from the development cycle • For example 6-month delivery cycle having 6-10 short iterations per delivery • Feedback can be integrated directly into each release • Web environment has many projects like this

35. Common mistakes • Project leader X: • Sure, we do not apply the waterfall, everyone knows that it does not work. We use Method X and are in our first project. • We are now into it for two months and the use caseanalysis is nearly finished. Then, we will plan and schedule what will be done in each iteration before the programming will start.

36. Text on success of agile methods • Read the text (4 pages) • What are the characteristics of the described projects? • What are the main characteristics of Agile development mentioned in the text? • Why are these methods successful?

37. The Agile alliance • http://www.agilealliance.com/ • Grouping of people active on Agile development (founded in 2001) • 4’200 members, many events, articles, publications available from the web side • Agile manifesto, and definition of main principles of Agile software development • Definitions described on their web pages

38. A little bit of history • Evo – evolutionary project management • 1960s: Evo and first elements of iterations etc. • 1986: SCRUM • 1996: eXtreme Programming • Mid 1990s: tendencies to more lightweight development as result of many failures • 2001: Agile alliance

39. The Agile manifesto • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan

40. The Agile principles (1) • 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. • 2. Welcome changing requirements, even late in the development. Agile processes harness change for the customer’s competitive advantage. • 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to a shorter time scale.

41. The Agile principles (2) • 4. Business people and developers must work together daily throughout the project. • 5. Build projects around motivated individuals. Give them the environment and support their need, and trust them to get the job done. • 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

42. The Agile principles (3) • 7. Working software is the primary measure of progress. • 8. Agile processes promote sustainable development. • 9. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. • 10. Continuous attention to technical excellence and good design enhances agility.

43. The Agile principles (4) • 11. Simplicity – the art of maximizing the amount of work not done – is essential. • 12. The best architectures, requirements, and designs emerge from self-organising teams. • 13. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.

44. Suitability of Agile methods • The culture of an organisation must be supportive of negotiation • People must be trusted • Fewer staff with higher level of competency • Organisation must live with decisions the developers make • Low level of hierarchies is better • Organisation needs to have a development that facilitates rapid communication between team members

45. A little break in the presentation … • Read the text exploring agile development • What are the two most important advantages of the waterfall approach and of Agile methods? What are the disadvantages?

46. So what exactly is Agile? • Not a single definition • Timeboxed, iterative, incremental development • Change is embraced • Promote evolutionary delivery • Simplicity, lightness, communication • Some methods common to a single practice • Common project room, standup meetings, pair programming, …

47. Classification of Agile methods • Length and number of cycles/iterations • From a few days to months • Ceremony or strict guidelines to follow • Formal steps, … • Suitability for particular sorts of projects • Projects with many/few developers • Projects with a long/short duration

48. Classification of four examples

49. Function points to measure project size • http://www.ifpug.org/ • FPU – Function Point analysis • Estimate size of a software project • Budget application development • Determine project productivity • Complex analysis in closed documents

50. Agile project management (Augustine et al.) • Guiding Vision: Establish a guiding vision for the project and continuously reinforce it through words and actions • Teamwork and collaboration: Facilitate teamwork and collaboration through relationships and community • Simple rules: Establish and support the team’s set of guiding practices such as SCRUM or XP