How to Start a Successful Program A Panel Discussion for Midwest Gateway INCOSE Chapter May 25, 2011

1. How to Start a Successful ProgramA Panel Discussion for Midwest Gateway INCOSE ChapterMay 25, 2011 In order of Appearance: Bob Scheurer Bill Jennings Shawn Simmons

2. “Successful Programs Begin with Good Planning” …But what is good planning???

3. Good Planning: • Combines the management approach with the technical solution definition/derivation • Encompasses the holistic view of planning and engineering for the complete system lifecycle • Continuous across pre-contract, post-award planning, and program execution

4. Good Planning Drives Execution • Successful Program Execution is dependent on • Clarity of program objectives and requirements • Clarity of activities, schedules, deliverables, resources • Early identification of needs • Viewing requirements in terms of the entire system life cycle • Integration of top-down needs and bottom-up solutions • Flexibility in response to changing conditions - updates All of these depend on good planning!

5. But Good Planning has Been Missing… • “A root cause of failures on acquisition programs includes …lack of technical planning and oversight ….” • Mark Schaeffer, OUSD AT&L. “The Value of Systems Engineering to Program Management.” INCOSE. DoD-directed Studies/Reviews, 2005. • “Reason for Weapon Systems Cost Growth is a poor foundations at Milestone B go ahead, including …. artificially low cost estimates and optimistic schedules and assumptions ….” • Memo from John J. Young (DoD USD AT&L) to DoD Secretary Gates, “Reasons for Cost Change for Selected Major Defense Acquisition Programs” (Feb 2009).

6. Could something else be missing, too?: Integration • Localized Technical Planning is Not Enough • Too Many Disconnected Program Visions • Inefficiencies from Plan Redundancies and Inconsistencies • Gaps in Planning Details • Integrated Technical Planning is the Gateway to Program Success • Organized and Disciplined • Aligned Planning Processes • Instituted at the Very Beginning of an Initiative • Describes Transformation from Problem (Requirements) to Design • On-Going Updates and Coordination

7. Integrated Planning Over the Acquisition Lifecycle Defense Acquisition Management System, May 22, 2009 extrapolated • The Materiel Development Decision precedes entry into any phase of the acquisition management system • Entrance criteria met before entering phase • Evolutionary Acquisition or Single Step to Full Capability User Needs Technology Opportunities & Resources Technology Development A B C IOC FOC (Program Initiation) Technological Maturity and Integration Risk Assessment Engineering and Manufacturing Development Materiel Solution Analysis Production and Deployment Operations and Support FRP Decision Review Competitive Prototyping PDR CDR LRIP/IOT&E Sustainment Pre-Systems Acquisition Systems Acquisition Materiel Development Decision Post-CDR Assessment Post-PDRAssessment = Decision Point = Milestone Review = Decision Point

8. Contract Contract CDRLs CDRLs SOW SOW SOO Lifecycle Phases Lifecycle Phases / Events … … Work Products Work Products SRR SDR PDR. SRR SFR PDR. SRR PDR CDR TRR SFR WBS / PBS • Work Product 1 Final Update Update • Work Product 2 Prelim Final Update • Work Product N - - - • Work Product 1 Final Update Update • Work Product 2 Prelim Final Update • Work Product N - - - Airframe Plan Maturation Organization WBS / PBS Pgm Plans IMP/IMS IMP/IMS WBS PEP WBS PM PEP PM Wing • Events • SignificantAccomplishments • AccomplishmentCriteria - - • Events RAM • Sig Accom (SA) - Acc Crit (AC) • Sig Accom (SA) • - Acc Crit (AC) Organization SEMP SEIT 1.0 SEIT SEMP 1.0 IPT 2.0 TEP TEP IPT 2.0 RM IPT 3.0 IPT 3.0 RM Gate D Gate C ATP IBR TEP Flt Flt 2.1 2.1 TEP 2.2 2.2 EE EE TEP TEP SW TEP 2.3 2.3 TEP SW SW Integration Lab System Integration Lab … … Standard Deck + Metrics Standard Deck Standard Deck Program Customized Plan (Basis for Proposal Estimate & Schedule) Plans Plans Pgm Plans Detailed / Refined Technical Plan Work Product Repository Detailed Functional Work Flows Outcome of Int. Tech. Planning Outcome is an Integrated Technical View

9. Elements of the Integrated Planning Integrated & Traceable • Statement of Objectives (SOO) / Statement of Work (SOW) (what to do) • Deliverables (evidence of work done) • Work Breakdown Structure (WBS) (how work is organized) • Program Technical Specification (technical requirements) • Specification / Document Tree (decomposition) • Program Plans (how work will be done) • Integrated Master Plan (IMP) – major events • Reviews and Audits • Systems Engineering Management Plan (SEMP) (what the supplier has planned) and Systems Engineering Plan (SEP) (what the acquirer/customer has planned) • Technical Performance Measures • Subordinate Plans (risk mgt, reqts mgt, trade studies, etc.) • Test and Evaluation Master Plan (TEMP) • Integrated Logistics Plan (for Operations and Support) • Use of Suppliers • Resource Requirements and Staffing Organization (how people are organized) • Metrics for Technical Progress, Technical Maturity, and Quality/Productivity (measurement and feedback mechanism) • Risks & Opportunities • Inputs to Earned Value Management System • Integrated Master Schedule (IMS) with Activity Networks (when) • Cost Estimates (resource management)

10. And the Planning Needs to Continue:Plan the Work / Work the Plan … for Each Phase A B C MDD Materiel Solution Analysis Engineering and Manufacturing Development Production and Deployment O&S Technology Development CDD CPD CBA ICD PDR CDR FRPDR Planning Executing

11. Overview Summary of Integrated Technical Planning • Performed Regularly Across the Total System Lifecycle • Encompasses a Diverse Set of Views to Form the Complete Planning Picture • Receives Guidance for Constructing the Various Plans from: • Customer Documents • Systems Engineering Process • Product Architecture (Product Components via Modeling) • IMP/IMS

12. Customer Documents RFP/BAA/ECP CLINs/ CDRLs/SDRLs Technical Plan Documents Customer SEP SOO/SOW SEMP & Supporting Technical Plans IMP & IMS Contract WBS Contract SOW PEP/TEPs/ Organizational Breakdown Structure Planning Packages Getting Started – Identifying the Planning Elements Methods & Tools Questions Frameworks Process Guides Process Networks Planning Checklists DAG BAA – Broad Agency Announcement BPI – Business Process Instruction BPG – Business Process Guide CLINs – Contract Line Item Numbers DAG – Defense Acquisition Guidebook ECP – Engineering Change Proposal IMP – Integrated Master Plan IMS – Integrated Master Schedule PEP – Program Execution Plan RFP – Request for Proposal SDRL – Supplier Data Requirements List SEMP – Systems Engineering Management Plan SEP – System Engineering Plan SOO – Statement of Objectives SOW – Statement of Work TEP – Team Execution Plan WBS – Work Breakdown Structure CDRL: Contract Data Requirements List Contract Deliverable Requirements List Contract Document Requirements List Contractor Data Requirements List Customer Data Requirements List

13. Iterate to align & add detail Technical Plan Documents –The Logical Connectivity Flow and Interfaces Between Planning Elements Technical Plan Documents SEMP & Supporting Technical Plans IMP & IMS Customer/Acquirer Contract WBS SOO WBS SOW Contract SOW PEP/TEPs/ Organizational Breakdown Structure Contractor Planning Packages CSOW CWBS Questions OBS  RAM Program Plans IMP Checklists Processes SEMP SA’s and AC’s Industry Guidance Program Specific Risks Planning Packages IMS

14. Summary of Program Planning Building Blocks BAA – Broad Agency Announcement CLINs – Contract Line Item (Numbers) CSOW – Contract Statement of Work CWBS – Contract Work Breakdown Structure DAG – Defense Acquisition Guidance EVM – Earned Value Management PBS – Product Breakdown Structure PEP – Program Execution Plan RFP – Request for Proposal SEMP – Systems Engineering Management Plan SEPMQC – SE Proposal Maturity Quality Checklist SOO – Statement of Objectives SOW – Statement of Work WBS – Work Breakdown Structure Build-up of Details Work Packages Work Packages Financials Financials Operating Plan Operating Plan EVM EVM Budget Labor/ Skills Budget Labor/ Skills Estimates Contracts Facilities Estimates Contracts Facilities Planning Packages Planning Packages Integrated Master Schedule Tiers 3 and Below Integrated Master Schedule Tiers 3 and Below Other Supporting Other Supporting PEP PEP SEMP Manuf , Support, Test Plans SEMP Manuf , Support, Test Plans Plans Plans Iterate these to add detail CSOW Organization Organization Processes CSOW CWBS Processes CWBS Integrated Master Schedule Tiers 1 & 2 Integrated Master Schedule Tiers 1 & 2 Integrated Master Plan Integrated Master Plan Foundation/ Inputs Planning Guidance Major Planning Guidance Major SOW/ WBS/ SOW/ WBS/ Success RFP/ BAA/ SOO/ Success RFP/ BAA/ SOO/ PBS PBS DAG/ Questions/Checklists Questions Questions Criteria Criteria CLINs/ Referenced Documents CLINs/ Referenced Documents Command Media Command Media & Products & Products 14 - Engineering - P. Mgmt & Engr - Customer - P. Mgmt/ Finance

15. Program Definition Documents • To identify Customer documents that indicate the program requirements for each program phase, examine the related Program Definition Documents. • Example: DoD Instruction 5000.02 ‘Operation of the Defense Acquisition System’ • http://www.dtic.mil/whs/directives/ • Defense Acquisition Management System • Material Solution Analysis, Technology Development, Engineering & Manufacturing Development, etc. phases

16. B C A Phases Production and Deployment Materiel Solution Analysis Operations and Support Engineering and Manufacturing Development Technology Development Prototyping System Design Sustainment Post-CDR Assessment Materiel Development Decision FRP Decision Review Post-PDR Assessment Disposal Program Reviews with a technical basis TRA TRA OTRR IBR IBR IBR AOTR PDR may occur after MS B for less than MDAP Technical Reviews SVR /FCA /PRR ASR TRR ITR CDR PCA SFR ISR SRR Prototype Technical Reviews PDR Initial Product Baseline Product Baseline System Functional Baseline Allocated Baseline System Design Specification Preferred System Concept Technical Baseline Technology Readiness Assessment Technical Reviews PRR - Production Readiness Review SFR - System Functional Review SRR - System Requirements Review SVR - System Verification Review TRA - Technology Readiness Assessment TRR - Test Readiness Review IBR - Integrated Baseline Review ISR - In-Service Review ITR - Initial Technical Review OTRR - Operational Test Readiness Review PCA - Physical Configuration Audit PDR - Preliminary Design Review AOTR - Assessment of Operational Test Readiness ASR - Alternative Systems Review CDR - Critical Design Review FCA - Functional Configuration Audit FRP – Full Rate Production Program Reviews Decision Point Milestone Reviews Decision Point if PDR is not conducted before MS B Purpose The Defense Acquisition Guidebook Focuses on the Technical Baselines and Maturing the Technologies/Designs Focus On Technical Baselines

17. Questions Framework Q # Questions Sub-Questions WHEN Sub-question in Inputs Lower Level Tasks Outputs Entry, Task Format Early, (Use Case During names) and 2 Does the program have documented 1. Documented resolution of capability needs at the system level conflicts, uncertainties, and (Capability Description Document or ambiguities within the equivalent) ? program and with the customer 2. Updated capability needs 2.1 Does the program have Entry Collect the 1. Possible sources -- 1. Identify the inputs Consolidated list of documented capability needs documented ORD/O&O, CDD, that may have capability needs at the system level? capability needs SRD, SPS, SOW, capability needs RFP (Sections L&M 2. Search the inputs and SOW) for capability needs 2.2 Have these capability needs Early Review the Consolidated list of 1. Review the needs Description of conflicts, been reviewed for capability needs capability needs 2. Identify conflicts ambiguities and understanding by the for understanding 3. Identify uncertainties program? by the program ambiguities 4. Identify 2.3 Have areas of potential Early Resolve the 1. Consolidated list of 1. Hold discussions 1. Documented resolution of misunderstanding or lack of conflicts, capability needs with the stakeholders conflicts, uncertainties, and clarity been identified and ambiguities and 2. Description of to resolve the ambiguities within the resolved for consensus within uncertainties with conflicts, ambiguities conflicts, ambiguities program and with the the program and with the the customer and uncertainties and uncertainties customer customer? 2. Update the 2. Updated capability needs capability needs LSI Owner(s) / PMUA / Event LSI Manager TRADOC SA AC Closure Criteria TPMs Process and Measures Updated Beyer / Shannon Strand / MAJ Byers 110 050 SoS/FoS TPMs updated and approved to reflect current program management monitoring needs as of SoSFR . Review and concurrence , subject to required changes, of the approved TPMs by appropriate Tier 2 & 3 One Team Management. Required changes implemented in a timely manner. SoS/FoS TPMs updated & approved to reflect current program management 110 050 1 needs Formal Guidance Question-based Framework By Stage Question-Based Construct By Stage • Program Reviews • JCIDS & DoDI 5000.02 • Questions • Sub-questions • Lower Level Tasks & I/O JFCOM DoD Questions IMP Significant Accomplishments & Accomplishment Criteria IMS Tasks by IPT • Program Reviews/ Events • Program Reviews/ Events • Critical Tasks and Work Products • Significant Accomplishments • Accomplishment Criteria IMS IMP 17

18. Planning information availability by phase Available Information Available Information • Concepts of Operation • Initial system-level capabilities • Operations analysis studies • Technology readiness information • Lifecycle cost estimates • Finalized system-level capabilities • Requirements • Functional architecture views • Physical architecture views • Preliminary design • Key supplier data • Substantiated lifecycle cost estimates Materiel Solution Analysis Engineering & Manufacturing Development • Viable set of system-level capabilities & requirements • Conceptual architecture views & design concepts • Trade studies supporting draft capabilities & requirements • Key supplier data • Detailed lifecycle cost estimates • Completed design • Complete product assembled and tested • Manufacturing and assembly process in place • Fully mature lifecycle cost estimates Technology Development Production & Deployment 18 18

19. Purpose of the Integrated Master Plan • Provides a framework for planning integration across the program • Represents the integrated product development encompassing all functional disciplines • Provides a common framework for developing the IMS and detailed team schedules IMP Event-based • Serves as a contractual agreement with the customer on what will be done • Documents program events (PEs) • Identifies significant accomplishments (SAs) for each event • Specify accomplishment criteria (ACs) for each Significant Accomplishment • Provides traceability to WBS, SOW, deliverables, risks, and opportunities 19

20. Purpose of the IMS IMS Calendar based • Framework for planning packages and ultimately work packages • Facilitate resource needs estimating and use planning • Track progress and completion status • Basis for determining progress against planned efforts including start/finish/duration • Key supplier events • Documented schedule of program events • Specify tasks to satisfy accomplishment criteria for each significant accomplishment • Task traceability to WBS, SOW, deliverables, & program milestones • Time-phased tasks and show predecessor and successor relationships • Time-phased integrated product development encompassing all program teams • Common framework for developing detailed team schedules 20

21. Translating IMP into IMS content Lesson 3 – Develop an IMS using PMA IMS IMP Sequential Flow on Vertical Axis Sequential Flow on Horizontal Axis Tier 1 – Events and tasks Tier 2 – Events and tasks Tier 3 – Events and tasks Tier 4 – Events and tasks • Program Events (PEs) • SAs and ACs Lower level tasks 21

22. Example: CWBS Dictionary Lesson 3: Relationships between Planning Elements 22

23. Systems Engineering Management Plan (SEMP) SEMP Program Overview & Objectives Document Scope Applicable and Reference Documents SE Organizational Integration and Technical Authority Systems Engineering Technical Processes SE Management Processes Integration with Program Management Efforts Transitioning Critical Technologies Integrated Environment for Systems Engineering Additional and Specialty SE Activities Supporting Plans Systems Engineering Management Plan (SEMP) describes the contractor's technical approach and proposed plan for the conduct, management, and control of the integrated systems engineering effort. 23

24. The structure of Integrated Technical Planning Program Events Define the availabilityof a Capability at a point in time. Accomplishments Represent requirementsthat enable Capabilities. Criteria Represent Work packages that fulfill Requirements. Work Package Work Package Work Package Work Package Work Package Work Package Work Package Work Package

25. Backups

26. Assessing Completeness Integrated Program Planning Achieve program technical objective • Discover and demonstrate related findings of the assessment • Apply the Questions Framework to get from the incomplete state to the complete state • Correct deficiencies SOW Requirements CDRLs Risks Processes = Executable Questions IMP/IMS Exit WBS EVM Criteria | 1 File name.ppt • Assess adequacy of integration of technical plans to ensure completeness • Identifying conflicts between plan elements • Identifying intermediate work products late or out of sequence • Identifying incompatible levels of detail • Identifying conflicts in the IMP/IMS

27. Purpose Steps 1 & 2: Determine the intended program state • Step 1: Collect customer provided descriptions of their needs and concepts • RFP/ BAA/ RFA etc. • JCIDS Functional Needs Analysis (FNA) • JCIDS Functional Solutions Analysis (FSA) • Initial Capabilities Document (ICD) • Analysis of Alternatives (AoA) • Capabilities Development Document (CDD) • Step 2: determine where your program is in the acquisition life cycle • Identify the Program Events (PEs), e.g. ITR, ASR, SRR, etc. • Identify technical objectives associated with events 27

28. Technical guidance is available for major program stages “Questions focus” Use Product Flow but add Questions philosophy if trying new methods & technologies Questions & Product Flow A top-Down, question-based constructdrives Integrated Planning by establishing a framework for robust, successful program execution & RFP responses