Improving Productivity on an SEI Level IV Project presented by Kelly Ohlhausen

1. Improving Productivity on an SEI Level IV Projectpresented by Kelly Ohlhausen

2. Agenda • Project Description - Where we were… • Why is it important? • How did we find the answers? • What did we find? • Where we are now… • Details of what we did • Summary

3. Where we were... • One of four projects evaluated during an SEI CBA-IPI Level IV assessment. (June 2001) • Productivity identified as a risk from the beginning (productivity = ELOC/hour) • Software bid was aggressive • 10% management challenge • 12% customer challenge • Real-time embedded system to be completed in two deliveries to the customer

4. Where we were...

5. Why aren’t we meeting the goal? • Was the productivity goal set unrealistically high? • Process - Are these the right processes? Are there too many? Too few? • Was the mix of engineering skills appropriate? • Was there enough training? • Software projects can’t ever reach goals so why bother setting them?

6. Why is this important? • Productivity estimates impact customer and management satisfaction, profits, and potentially future business. • Our customers need to be able to trust the numbers we use in a bid. • Profit margins disappear when projects are bid with unrealistically high productivity levels.

7. Finding the Answers • Initiated a Raytheon Six Sigma project to determine what, if any, actions could be taken to improve productivity during the implementation stage • Raytheon Six Sigma is an iterative knowledge-based process for improvement

8. Analysis Factors • Attrition • Learning curve (training new people) • Experience level • Hardware availability • Overlapping stages • System engineering • Customer directed slowdown

9. Information Sources • Metrics: size, cost, schedule, delivery, staffing, defect containment, requirements stability • Earned value progress metrics, audits, action items • Lessons Learned from prior stages • Two other projects using the same processes • R6Sigma Defect Containment Metric Analysis project results

10. Information Sources • Meeting minutes (e.g. weekly progress, metrics analysis) • Inputs from engineers and managers

11. Information Sources The search for information took place during R2 object design stage. • Software Development R1 R2 • - Requirements Analysis  • - System Design  • - Object Design X • - Implementation  • - Software Integration & Test X • - System Integration & Test • - Acceptance Test

12. Requirements for Application • Changes were to be put into place immediately. • The R2 implementation stage schedule had to be shortened by four weeks without removing scope to compensate for a schedule slip. The software integration and test stage had to start on time. • Changes could not have a negative impact on quality.

13. What we found... Moving toward SEI Level V (Optimizing) with a number of smaller actions involving defect containment and process change management could significantly impact productivity.

14. Where we are now... 34.3% increase * as of 2/1/02

15. Where we are now... Earned Value Metrics * as of 2/1/02

16. What was changed... • Escaping defects • What should be covered by a unit test was addressed in detail during the implementation stage training (JIT) • Stress was placed on pushing boundaries and not relying just on nominal values. More special case unit tests were used.

17. What was changed... • Escaping defects (cont.) • Run the unit tests on the target processors during implementation. Don’t wait until integration to discover a platform issue. • A step was added to the stage process for extra testing to identify rogue pointers before unit testing The full impact of these changes will not be known until integration and testing stage.

18. What was changed... • Developed the code within the configuration management tool from the beginning. Gave people a chance to learn the tool in small steps instead of all at once. • All tools and hardware identified and checked out BEFORE the stage began (matching versions across test lab, compiler updates in place, instructions available)

19. What was changed... • We did better defining the earned value progress metrics • Made sure there was enough detail (stage process steps were defined at the correct granularity) • Used realistic weighting Because… An incorrect plan yields misleading information. Time is wasted investigating non-issues or not identifying a problem in time.

20. What was changed... • Did better at following the standards and processes. • Invited the right people for the review. • Assigned roles to focus the review, esp. facilitator • The implementation stage Just-In-Time (JIT) training was used to remind the team of these issues. Because… • Products that do not meet standards have an effect on time during reviews as well as rework.

21. Summary • There was no conclusive evidence that the productivity goal was unrealistic. • Relatively small changes made an impact. • The research uncovered improvements for every stage. • If we had applied these improvements from the beginning of the project, we can only imagine what the overall productivity would have been.

22. Conclusion • Take time to look at what you are doing. • Learn, not just from your own experience but from the experience of others. • Continuously try to identify optimization opportunities. • Improvement can be made, even on an SEI Level IV program.

23. The greatest obstacle to discovering the shape of the earth, the continents and the ocean was not ignorance but the illusion of knowledge. Daniel J. Boostin The Discoverers