Why Revitalize Systems Engineering? or How We Got To This Point Today

1. Why Revitalize Systems Engineering?orHow We Got To This Point Today July 21, 2004 Tom Hoog Dayton Aerospace, Inc. © 2004 Dayton Aerospace, Inc.

2. Systems Engineering Eulogy • SE born in the 1930s as one of RCA Broadcasting Services & Standards • Military development of Ops Research in the 1940s was instrumental in shaping its youth • First formal education attempted at MIT in 1950 • SE became the darling of NASA & military systems in the 1960s; processes imposed on prime contractors • Its many offspring were Chief Systems Engineers, Directors of Engineering, VPs of Systems Engineering as well as many Systems Engineers • It became bloated & unwieldy in late 1960s; became known as AFSCM 375-5 • Attended fat farm & reappeared as Mil-Std-499A; grew in stature & acceptance • SE became check & balance in massive engineering efforts • It suffered an identity crisis when universities taught industrial engineering as systems engineering • In the 1980s SE attempted to re-identify itself as Mil-Std-499B • Later cashiered out of the military, it found civilian employment as an IEEE standard • It wasted away with an illness brought on by rejection of its processes & bastardizing of its techniques & methods • Design reviews became re-design meetings • SE died a lingering death in the 1990s & was replace by IPTs, Design Teams & Group Gropes • SE is survived by & remembered by a cult of old systems engineers who still practice their now meaningless rituals • SE was almost 70 years old at the time of its death • No services are planned Full text of this eulogy can be found at http://acc.dau.mil/simplify/ev.php?ID=6174_201&ID2=DO_TOPIC

3. Outline Background • DoD Transformation • 5000 • JCIDS • Scope of SE • Systems Engineering Revitalization Initiatives • USD (AT&L) Imperatives • SAF/AQ Initiatives • USD (AT&L) Initiatives • Health of SE – DAI’s View (Jul 03)

4. Outline Background • DoD Transformation • 5000 • JCIDS • Scope of SE • Systems Engineering Revitalization Initiatives • USD (AT&L) Imperatives • SAF/AQ Initiatives • USD (AT&L) Initiatives • Health of SE – DAI’s View (Jul 03)

5. BackgroundDoD Decision Support Systems • Three parts to DoD Decision Support Systems • Requirements Generation • Governed by CJCSI 3170.01 • reissued 24 June 2003 • Defense Acquisition • Governed by 5000 series • Reissued 12 May 2003 • Financial Management • Governed by multiple sources • Being revised Requirements Generation System Defense Acquisition System Financial Management System

6. Background Strategic Policy Guidance JCIDS Analysis Joint Operations Concepts Joint Operating Concepts Joint Functional Concepts Integrated Architectures Functional Area Analysis CPD Functional Needs Analysis CDD Post Independent Analysis Materiel Changes CJCSI 3170 process Alternative N Ideas for Materiel Approaches Analysis of Materiel Approaches DOTMLPF Analysis ICD Alternative 2 Alternative 1 DOTMLPF Change Recommendation DOTMLPF Changes CJCSI 3180 Process Functional Solution Analysis

7. THE “New” 5000 MODEL C A B User Needs & Process entry at Milestones A, B, or C Start into Increments where appropriate Increments are interrelated Continues for development life of system Technology Opportunities Pre-Systems Systems Acquisition Acquisition Sustainment (Program FOC IOC Initiation) Technology System Development Concept Refinement Production & Operations & & Demonstration Deployment Development Support FRP Design Concept Decision System Demonstration Readiness System Integration Decision LRIP/OT&E Review Review C MATURE B DEMO ADAPT INTEGRATE TEST AND FIELD MATURE B C DEV DEMO ADAPT INTEGRATE TEST AND FIELD B C DEV DEMO ADAPT INTEGRATE TEST AND FIELD MATURE Spiral Development - Insert technology into program when it is mature Adapted From DoDI 5000.2, 12 May 03

8. SE in Evolutionary Acquisition Environment • Multiple concurrent increments lead to increased need for disciplined SE process (vs. single step process) • Continuous requirements definition & technology maturation tasks • Complete definition of requirements • Plan to mature requirements • Traceable from top level operational requirements to design implementation • Inclusion of joint requirements (FoS, SoS, net–centric) • Allocation of requirements to increments • Mature producibility, affordability, supportability, of technology as well as performance   • Need to cover both government & industry aspects © 2004 Dayton Aerospace, Inc.

9. 5000 Series Extracts • DoDD 5000.1 • Para E 1. 27 Acquisition programs managed through application of SE approach that optimizes total system performance & minimizes total ownership cost. A modular open systems approach shall be used • DoDI 5000.2 • Para 3.9.2.2 Sustainment uses SE methodology • Para E 7.2 Human factors is part of SE • Para 7.7 Include ESH in SE • DoD 5000.2-R (Interim Defense Acquisition Guidebook) Para C2.3.2 Acquisition strategy includes SE Para C2.8.2 Support strategy is part of SE Para C2.8.5.5 Human factors is part of SE Para C2.8.6 ESH is part of SE Para C6.7.2.7 Critical Program Information is part of SE Para C5.2 18 pages on SE

10. DoD 5000.2-R, Para C5.2 • COTS • Reliability, availability & maintainability • Human systems integration • Environment, Safety & operational health • Interoperability • Survivability • Mission assuredness • Information assurance • Anti-tamper provisions • Other Design considerations • WBS • Performance Specifications • Metric system • Insensitive munitions • Value engineering • Precise time & time interval • Accessibility • Corrosion prevention & control • Defines what SE does • Defines the SE activities • Requirements analysis • Functional analysis/allocation • Design synthesis & verification • System analysis & control • Manufacturing & production • Modeling & simulation • Quality • Acquisition logistics • Open systems design • Software management • Other Design Considerations • WBS • Performance specifications • Metrics • Insensitive munitions

11. Systems Engineering Scope of Systems Engineering Acquisition community participation in early requirements process to assist in understanding Acquisition Requirements Adapted from DoDI 5000.2 12 May 2003

12. SE Definitions • Systems Engineering (SE) A comprehensive, iterative Technical Management (TM) process that includes translating operational requirements into configured systems, integrating the technical inputs of the entire design team, managing interfaces, characterizing and managing technical risk, transitioning technology from the technology base into program specific efforts, and verifying that designs meet operational needs. It is a life cycle activity that demands a concurrent approach to both product and process development. (11th Glossary of DoD Acqu Terms) • Systems Engineering - An inter-disciplinary approach and means to enable the realization of successful systems. (EIA 731.1)

13. Some Systems Engineering Characteristics • Encompasses government and contractor efforts • Either may lead • Some joint • Iterative process at multiple levels • System level • Subsystem/component level • Specific tasks depend on program phase • Pre-Acquisition, Acquisition, Sustainment • Specific focus of tasks/trades depends on type of program • New, modification • Prime or support equipment; system or subsystem • Aircraft, space, electronics, armament • Complete system, subsystem, System of Systems / Family of Systems © 2004 Dayton Aerospace, Inc.

14. System Engineering Focus Shifts over Program Life Cycle • Pre-Acquisition • Operational & system performance requirements • Life cycle affordability requirements • Acquisition • Design requirements & implementation • Manufacturability, supportability, upgradeability, etc. • System integrity, • Sustainment • Supportability • System integrity Modification/Upgrades: Repeat the Cycle © 2004 Dayton Aerospace, Inc.

15. Requirements Analysis Analyze missions & environments Identify functional requirements Define/refine performance & design constraint requirements System Analysis & Control Input Customer needs/objectives/ requirements Technology base Outputs from prior phase Program decision requirements Requirements applied through specifications & standards Trade-off studies Effectiveness analyses Risk management Configuration management Interface Definition Performance based progress measurements IMP IMS Technical reviews Requirements Loop Functional Analysis/Allocation Decompose to lower level functions Allocate performance & other limiting requirements to all functional levels Define/refine functional interfaces Define/refine/integrate functional architecture Design Loop Synthesis Transform architectures Define alternatives system concepts, configuration items & system elements Define/refine physical interfaces Select preferred product & process solutions Verification Output Phase dependent Decision support data System architecture Specifications & baselines Adapted from Mil-Std-499B (Draft)

17. Outline Background • DoD Transformation • 5000 • JCIDS • Scope of SE • Systems Engineering Revitalization Initiatives • USD (AT&L) Imperatives • SAF/AQ Initiatives • USD (AT&L) Initiatives • Health of SE – DAI’s View (Jul 03)

18. USD (AT&L) Imperatives • “Provide a context within which I can make decisions about individual programs.” • “Achieve credibility and effectivness in the acquisition and logistics support processes.” • “Help drive good systems engineering practice back into the way we do business.” From Feb 04 OSD (AT&L) - NDIA SE Division Meeting

19. SAF/AQ SE Revitalization • SAF/AQ Dr. Sambur Memo, Apr 9, 2003 • Focus attention on application of SE & elevate these disciplines commensurate with cost & schedule • Develop SE performance incentives • Include status of SE in future program reviews • Identify key SE processes & practices in acquisition documentation • Acquisition Strategy Panel briefings must address significant SE areas • SAF/AQ Memo, Increment 2, Jan 7, 2004 • Intended to institutionalize key attributes of an acceptable SE approach • Insert the appropriate language into the following acquisition documents • Solicitations • Award Fee Plan/Incentive Fee Contract • Contracts

20. USD (AT&L) Initiatives • AT&L SE Policy Memo, Feb 20, 2004 • Effective immediately; included in next DoD 5000 revision • All programs, all ACATs shall apply a robust SE approach • SEP for MDA approval • Drive good SE back into the way we do business • AT&L SEP Guidance Memo, Mar 30, 2004 • Purpose of SEP is to lay out a plan to guide the technical aspects of an acquisition program • Living document, tailored to the program; roadmap to support program management • No prescribed format; address integration of technical aspects to include: • The systems engineering processes to be applied in the program • The systems technical baseline approach • Event-driven timing, conduct, success criteria, and expected products of technical reviews • The integration of systems engineering into the program’s IPTs

21. Outline Background • DoD Transformation • 5000 • JCIDS • Scope of SE • Systems Engineering Revitalization Initiatives • USD (AT&L) Imperatives • SAF/AQ Initiatives • USD (AT&L) Initiatives • Health of SE • Briefing to AT&L (SE OPR) • Quick Study

22. Health of SE Tasks • DoD 5000 & JCIDS reissued in May & June 03 • Briefing in Jul 03 • DAI personnel experiences • Lack of execution discipline on programs • 18 problem areas cited • Common Denominators • List of Issues • Candidate emphasis areas listed • DAI asked to provide supporting data • Quick Study Report/Briefing in Sep 03

23. Common Denominators • Program Teams are incentivized by cost and/or schedule - not execution of disciplined SE practices • Products and Processes are getting out of balance • Inadequate depth in applied SE capability © 2004 Dayton Aerospace, Inc.

24. Issues Lack of SE Execution Incentives • Contracts reward cost and schedule performance Product and Process Imbalance • Emphasis on speedy delivery of product can De-emphasize “processes” • Misapplication of spiral / evolutionary development intent is used to justify program shortcomings – “fix it in the next spiral” • Lack of true multifunctional activities in IPTs © 2004 Dayton Aerospace, Inc.

25. Issues (cont) Inadequate Depth in Applied SE Capability • Sound SE execution requires many trade studies • Government and industry “de facto” rely on their personnel to pass along “lessons learned” • Government and their contractors are losing “critical mass” in some areas • Systems Engineering processes are adequately defined BUT not followed, not understood, or are short circuited • Advertised CMMI ratings do not assure effective program implementation © 2004 Dayton Aerospace, Inc.

26. Summary of Observed Problems • Observed Problems - Approach • Three Categories of Problems • Requirements Management (8) • Systems Engineering Process (7) • Engineering Management (6)

27. Observed Problems - Approach • Based on analysis of many programs • Each problem area included • Observed Problem/Issue • Statement of problem or issue • Program examples • Impact • What happened because of problem • Recommendations • What can be done to avoid the problem • Focus was on systems engineering - recognized that systems engineering execution is not independent of overall program management • Objective was to improve overall program performance © 2004 Dayton Aerospace, Inc.

28. Observed Problems Requirements Management • Incomplete Definition of Requirements • Requirements Creep • Requirements Volatility • Lack of a Specification Tree • Incomplete/Weak Verification requirements • Late requirements Forced Into Software • Interface requirements and Interface Management • Failure to Implement and Maintain Requirements Traceability Tools © 2004 Dayton Aerospace, Inc.

29. Observed Problems (cont) Requirements Management Example • Inadequate definition of requirements • Statement of operational requirements • Statement of contractual performance requirements • Problems • ORD too detailed; includes solution • SRD restates ORD • Contract spec paraphrases SRD; includes design solution • Requirements not refined at lower tiers; incomplete requirements to subcontractors • Incomplete requirements analysis • Impacts • Design & implementation cannot proceed effectively • Incomplete design; scrap & rework; expectations not met • Recommendations • Enforce requirements reviews at Milestone B Review • Establish process to assure the SRD is complete & accurate • Reinforce requirement that IMP have well defined accomplishments with entrance & exit criteria; SRR must be key event for “opening the gate” to proceed with design • Develop templates, guides checklists to aid in review processes © 2004 Dayton Aerospace, Inc.

30. Observed Problems (cont) Systems Engineering Process • Systems Engineering Process Not Defined on the Program • Systems Engineering Process Defined But Not Applied • Lack of Flow Down of Systems Engineering Process Requirements to Development Subcontractors • Lack of Robust Systems Engineering Applied to Top Level System Design • Lack of Timely System Integration and Test (I&T) Planning • Inadequate Planning for Obsolescence and Sustainment • Design Incorporated into Performance Specifications © 2004 Dayton Aerospace, Inc.

31. Well Documented Industry Standard Very Generic Corporate Process Company Approach Top Level Applicable to all Products Division Process Applicable to Product lines Rarely Documented Program Process Applicable to Specific Program Without a Tailored & Documented Systems Engineering Plan, Program Execution Suffers

32. Why a Program Specific SEP • Directly tied to rest of program • Consistency across program plans & documentation (SOW, WBS, EVMS, etc.) • Readily correlated to IMP & IMS for execution • Establishes technical approach for stakeholders • Includes program unique nuances • Established by program requirements • Provides for explanation of unique tailoring © 2004 Dayton Aerospace, Inc.

33. Observed Problems (cont) Engineering Management • Inadequate Schedules and Budget to Accomplish Development Effort • Assumed Reuse Not Confirmed • Lack of Active Engineering Management of Development Subcontractor • Integrated Master Plan (IMP) / Integrated Master Schedule (IMS) Not Adequately Addressed and/or Not Followed on the Program • No Single Technical focal Point (Chief Engineer) © 2004 Dayton Aerospace, Inc.

34. Conclusions • Execution of sound SE processes is more important than ever • Need forcing functions that drive execution of disciplined SE processes Sound SE execution needs to be key ingredient in contract incentives © 2004 Dayton Aerospace, Inc.

35. Questions