INF5181: Process Improvement and Agile Methods in Systems Development

1. INF5181: Process Improvement and Agile Methods in Systems Development Lecture 08, 3.11.2011: Software process improvement in practice: Estimating software development effort Bente Anda Senior advisor, Norwegian Tax Administration Adjunct associate professor, IFI email: bentea@ifi.uio.no Fall 2011

2. Software process improvement (SPI) • A process is a series of steps, that is, what we do, to produce a result • A software process is the set of activities, methods, and practices used in the development and maintenance of software • This lecture focuses on one sub-process, the estimation process • Software process improvement is change of some kind to the way software development and maintenance is conducted • Organizations do software process improvement to reduce defects, improve productivity, lower costs… • But, changing work habits is difficult, and SPI frequently fails…

3. Software development at Skatteetaten Smidig / Scrum UP/UML SMART • Appr. 320 people working on software development as well as a large number of consultants • Appr. 12 ongoing software development projects • Several hundred change requests to existing systems each year • Methods: Project management Software development PMBOK

4. Effort estimation • Definition of estimate: Most likely effort to accomplish a software development task • Estimates of software development effort are necessary to manage scope, effort and resources Scope Time Resources • A challenge of defining an estimation process and methodology to improve effort estimation is that estimates often contain an element of magic X?

5. Effort estimation dilemma An effort estimate is often produced early when there is a large uncertainty with respect to the software product and software development project An estimate may have a large impact on our priorities (and consequently on what we deliver) and on how well we succeed in delivering it

6. The role of effort estimates Process for estimation Estimates Management (prioritization and descision) Project Portefolio scope time resources strategy/goal time resources Predictability resources Predictability delivery Predictability scope Control strategy/goal

7. Motivation for software process improvement within effort estimation • Not all software development is estimated • The estimates are not always used in prioritization, resource allocation and planning • Estimates often have a very large risk buffer • Little documentation of prerequisites and assumptions for estimates :

8. Motivation cont. • Too high estimates mean • unnecessary costs since estimates of time and resources tend to be fulfilled, and • productivity is too low • Too low estimates mean • other projects/change requests that would have given better cost-benefit are not prioritized, and • additional costs when consultants must be hired to “save” projects • Given the size of the organization, consequences are high

9. Stages of SPI – the IDEAL model IDEAL SEI`s approach to software process improvement

10. Keys to successful SPI • Define and communicate the problem to be solved through the SPI initiative throughout the organization • Align the SPI initiative with business strategy and goals • Be sensitive to the organizational context • Gain commitment from key stakeholders • Dedicate resources to manage the implementation and coordinate the process improvement activities • Invest in the necessary tools/infrastructure • Document key metrics before and after the process change • Publish evidence of SPI success • Learn from the feedback results • Be aware that SPI is change management

11. Challenges of SPI Technological challenges: Many methods, techniques and tools for effort estimation to choose from Organizational challenges: Many stakeholders Complex related processes that frequently change How to measure improvement

12. Very diverse tasks need to be estimated Types of task: Small change requests, user stories… large projects Requirements may be known or uncertain Development platform may be familiar or unknown Resources may be known or uncertain … Technology is complex and a moving target, and there are many estimation techniques: Estimation by analogy Parametric estimation Use case based estimation Wideband Delphi Successive estimating Bottom-up estimation based on a work breakdown structure Planning poker … Challenges of SPI - Technological Which technique should be used when?

13. Challenges of SPI – Organizational • Skills • ”I don`t know these techniques, so how can we use them?” • The nature of effort estimation • “An effort estimate is nothing more than a guess anyway”, and • “A number can be guessed very easily without any technique, so why should I bother to use a technique” • Measurement – • “We will not reveal bad practices”

14. Stages of this SPI project Motivation Goals Sponsorship Infrastructure Baseline Development of methodology Best practice Iteration 4 Iteration 1 Iteration 3 Iteration 2 Measurement Pilot 1 Training Pilot 2 Pilot 3

15. Stages of SPI project - Initiating Motivation Stimulus for improvement: Gartner report  Set goals for the improvement Establish sponsorship Establish improvement infrastructure Goals Sponsorship Infrastructure

16. Set goals Financial Non-financial • Less need to hire additional resources to ”save” projects • Less effort required to produce estimates • Estimates of higher quality • Less need for re-planning caused by budget overruns • More satisfied employees Quantifiable • Better investments caused by better prioritization • Better use of resources • Better possibilities for assessing offers from external suppliers • Better reputation in the Department of Finance • Improved predictability with respect to project completions • Improved understanding of uncertainty in estimates and of risk • Improved project management through • Improved understanding of cost drivers • Routines for collecting effort data throughout the projects • Documented practice that enables continuous improvement Non- quantifiable

17. Establish sponsorship Improved strategic management and more efficient deliveries of IT services • The SPI project was part of large project to renew IT function • Goals • Reference group with: • Member of IT staff • Project director • Manager of architecture team • Senior manager Free resources Simplified and renewed portfolio of systems More efficient maintenance and evolution, right quality on service deliveries Increased capacity for development Realize Reduced development time Reduced costs

18. Establish improvement infrastructure • Responsibles for the SPI initiative: • one senior project manager • one expert on SPI and effort estimation • Wiki

19. Stages of SPI project - Diagnosing Baseline Best practice Establish a baseline of current practice for effort estimation Develop recommendations based on best practice

20. Techniques used in SPI • Interviews • Interviews are used to access to tacit knowledge • We used interviews as a primary means to gather a baseline and information about best practice of effort estimation in Skatteetaten and in comparable organizations • Workshops • Workshops are conducted to do some work and create a product • We used a workshop to • describe best practice from the interviews and the collected material • make the first version of the estimation methodology • Literature reviews • Literature reviews are conducted to establish evidence-based best practice • We used a literature review to identify relevant best practice

21. Interviews Interviewees: Project and group managers with extensive experience from estimating software development effort (internal and from a number of external organizations) Duration: 1.5 hours Semi-structured interviews Interview guide: What are the estimation units? What is input to estimating? Who are responsible for estimating and who participates? Which estimation techniques and tools do you use? Do you have historical effort data? How are earlier experiences with effort estimation used? When do you estimate and re-estimate? How do you document the estimates? Positive and negative experiences with existing estimation practice?

22. Workshops • Participants: • Facilitator, documenter (SPI team), IT staff, project manager, 2 managers, 2 developers (all very experienced) • Participants should be chosen according to their influence in the organization, their skills and interest in the topic, and to what extent they will be impacted by the outcome of the workshop • Duration: 5 hours • Preparation: • Informed about focus of workshop and expected results • Distributed all material from interviews • Room: Walls for notes

23. Best practice for effort estimation at Skatteetaten-excerpt

24. Literature review • Identify, evaluate and put together relevant studies to one whole • Protocol for literature review • Select relevant databases for search (www.scholar.google.com) • Search for studies (Best practice software effort estimation) • Determine their quality • Put together the relevant information from the studies of sufficiently high quality

25. General best practice for effort estimation Understand the functional size of the software task to be estimated (the most important cost driver) Understand the uncertainty in product and project and include this in the estimate Use group estimation Combine estimates produced by different people and/or by using different technique and look at where they are different and where they converge Check that the estimate takes all requirements (also the non functional and implicit requirements) in to account and includes all necessary activities Separate estimate from expectations regarding resources 25

26. Stages of SPI project - Acting Development Testing Development of the methodology in four iterations Testing of the methodology in pilot projects Establish measurement scheme Measurement

27. Ideas Prioritization Project Realizing benefts Development Different estimation processes at different stages Earlyestimates Planning estimates Re-estimation

28. Development cont. • Requirements specification available • Development platform and resources known • Estimation technique: • Bottom-up based on work breakdown structure • Early project suggestions • Estimation techniques: • Wideband delphi • Successive estimating • Use case based estimation Early estimates Planning estimates Re-estimation Task estimates • Small tasks with low uncertainty • Estimation techniques: • Planning poker • Checklist based

29. Development Configuration of estimation processes Analyze project Choose estimation process and techniques Choose project methodology and software development methodology Historical effort data Estimation template ExperienceDB Adapt work breakdown structure Log estimation data WBS

30. Design of Wiki (v0.1)

31. Piloting • Workshops where Planning Poker was used to identify important cost drivers for early estimation • Adapted templates for early estimation • Tested techniques and templates for effort estimation and re-estimation, particular focus on Wideband delphi • Tested routines for collecting historical effort data, including use of estimation units according to RUP

32. Examples of tasks to be estimated with PP to identify cost drivers • Task 1: Establish connection between single owner company and owner in a specific time period • Task 2: Make new star for MVA to access historical transactions • Task 3: Make it possible to access 3 tables from a known source in the control mart.

33. Measurement

34. Effort estimation is a starting point of software development and effort estimates are vital in prioritizing and planning There are many techniques for effort estimation SPI is often difficult as there are both technological and organizational challenges Technical understanding, alignment with business goals, understanding of the relevant organizational processes, and commitment and involvement from stakeholders is important Models and frameworks for SPI may serve as tools for planning your own SPI initiatives To learn more about effort estimation: Summary