Working Group Meeting

1. Working Group Meeting Ricardo Valerdi Thursday October 27, 2005 Los Angeles, CA 20th International Forum on COCOMO and Software Cost Modeling

2. Agenda 8:15 AM COSYSMO Status 8:45 AM Major updates COCOMO/COSYSMO Overlap COSYSMO book outline Exercise in Psychometrics COSYSMO game Systems engineering effort estimation in O&M phase 9:15 AM SE Staffing profile survey 9:45 AM Break 10:15 AM Ongoing projects Data availability for COSYSMO (Chris Miller) Schedule implications in COSYSMO (Anthony Peterson) COSYSMOstar (Dan Ligett) 11:45 AM Lunch

3. COSYSMO Evolution Phase 1: Baseline ☺ Phase 2: Prototype ☺ Phase 3: Validate ☺ Phase 4: Institutionalize

4. Notional Estimation Example Company “Lockheed Grumman” is developing a system that has: 100 easy, 50 nominal, 75 difficult requirements 2 easy, 3 difficult interfaces 4 easy algorithms 5 nominal operational scenarios Size Drivers 36 Person Months of systems engineering effort COSYSMO Effort Multipliers Calibration High requirements und High tech risk High process capability

5. COSYSMO Status academicCOSYSMO, myCOSYSMO, COSYSMO Risk Add-on, and COSYSMOstar continue to be improved and can be downloaded from www.valerdi.com/cosysmo

6. Major Updates • COCOMO II/COSYSMO overlap • MBASE/RUP phases and ISO 15288 • Integration/test/requirements activities • Calibration factor clarification Hours = 38.55 * (size)^1.06 * product (EM) PM = 38.55/152 * (size)^1.06 * product (EM) 0.25

7. COSYSMO Book Outline • Scope of COSYSMO • Background on Systems Engineering • COCOMO II and COSYSMO • Estimation example • Model definition • Model form • Size drivers & counting rules • Cost drivers & interpretations • Model verification & validation • Statistical tests • Model parsimony • Bayesian approximation • Model usage • Experience base • Model Tailoring • Institutionalization & training

8. Exercise in Psychometrics* Psychometrics is the field of study (connected to psychology and statistics) concerned with the measurement of "psychological" aspects of a person such as knowledge, skills, abilities, or personality. • Stakeholder team cohesion • Personnel/team capability • Personnel experience/continuity • Process Capability *Inspired by Clark and subsequently Guilford Source: http://en.wikipedia.org/wiki/Psychometrics Source: Guilford JP. Psychometric methods. 2nd edn. New York: McGraw-Hill; 1954.

9. Reality Check

10. Stakeholder team cohesion Represents a multi-attribute parameter which includes leadership, shared vision, diversity of stakeholders, approval cycles, group dynamics, IPT framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in stakeholder community of the end users, customers, implementers, and development team.

11. Personnel/team capability Basic intellectual capability of a Systems Engineer (compared to the national pool of SEs) to analyze complex problems and synthesize solutions. Personnel experience/continuity The applicability and consistency of the staff at the initial stage of the project with respect to the domain, customer, user, technology, tools, etc.

12. Process capability The consistency and effectiveness of the project team at performing SE processes. This may be based on assessment ratings from a published process model (e.g., CMMI, EIA-731, SE-CMM, ISO/IEC15504). It can also be based on project team behavioral characteristics, if no assessment has been performed.

13. COSYSMO Game Purpose: to serve as a training tool for stakeholders to understand the capabilities and limitations of COSYSMO Rules The formal part of the game. A game designer designs the rules of the game directly but designs the player’s experience only indirectly. Culture Simulated project which realistically mimics the real world of systems engineering. Play Rule-bound and free-form; Players are assigned roles and given a project. Source: Rules of Play: Game Design Fundamentals by Salen & Zimmerman, MIT Press 2003.

14. COSYSMO Game • Rules • Facilitator sets the stage for a “case study” • Deliberately creates a model clash scenario • Play • Role-based decision making • Culture • Expectations and politics must be highlighted • Bring forward social process of cost estimation

15. Cast of Characters

16. Role Playing Activities Source: Models of Teaching, 6th Ed., by Joyce, Weil, Calhoun, 2000.

17. Role Playing Activities Source: Models of Teaching, 6th Ed., by Joyce, Weil, Calhoun, 2000.

18. Conceptualize Operate, Maintain, or Enhance Replace or Dismantle Transition to Operation Oper Test & Eval Develop Operate & Maintain Phase Current scope Suggestions: • Need to clarify point where M&E begins • Add “deleted” category (may be a different weighting) for requirements, interfaces, etc. • Consider that not all current COSYSMO parameters will apply here and that the ones that do, may have different values • Need to consider business strategies with respect to OM&E (e.g., problem report-driven, level of effort/priority driven, requirement changes, upgrade/technology refresh activities, etc.)—may require other size drivers • Need to better define OM&E activities • Consider frequency of deliveries as a driver

19. Inconsistent Effort Reporting • Data was adjusted based on the following: • Effort distribution across • EIA632 processes • ISO15288 phases • Yielding

20. Effort Distribution Across EIA 632 Fundamental Processes N = 18 Total = 100%

21. EIA/ANSI 632 Effort Profiling mini-Delphi Transition to Operation Operational Test & Evaluation Conceptualize Develop ISO/IEC 15288 Acquisition & Supply Technical Management System Design Product Realization Technical Evaluation

22. N = 15 Effort Distribution of EIA 632 Fundamental Processes Across ISO 15288 Phases In each cell: Average (Standard Deviation)

23. Contact Ricardo Valerdi MIT Lean Aerospace Initiative rvalerdi@mit.edu (617) 253-8583 www.valerdi.com/cosysmo