Air Force Systems Engineering Focus

1. Air Force Systems Engineering Focus NDIA Systems Engineering Division Meeting 18 Aug 2010 Col Shawn P. Shanley SAF/AQRE 703.254.2487 shawn.shanley@pentagon.af.mil

2. Agenda • What WSARA & Related OSD Initiatives Mean to AF SE • Leadership Perspective / Expectations / Challenges • AF SE Conference • What AF is Doing to Focus Our Practitioner Community • Process Focus • Development Planning / Early System Engineering • AF Program Support Review (PSR) (integrated reviews) • Policy Focus • Recent Policy Changes • Near Term Policy Changes • Specialty Areas (RAM, HSI, ESOH, etc) • Specs/Standards & RAM • People Focus • Required Resources

3. Bottom Line Up Front –What WSARA Means to the AF • AF understands the need to improve our acquisition approach • SECAF/CSAF initiated AF Acquisition Improvement Plan prior to WSARA • Forward-looking AF response to OSD’s WSARA-based questions • Grounded in acknowledgement that “all is not well” • Focused on what we are doing about it • “Business as usual” policies and processes won’t solve the problems • Innovative policies / processes in work through various AF initiatives • Additional workforce alone is wrong approach • Need right number of personnel with the right skill sets to support policies and processes • Change doesn’t happen overnight • Continued leadership support and commitment to improvements and visible ROI over time helps ensure we realize the improvements $295 Billion in Overruns & Major Schedule Slips – How SE Can Help to Solve the Problem!

4. Leadership Perspective • Robust SE contributes to successful programs • Front end trades • Cost budget analysis • Performance vs. cost and/or schedule • SE focus areas to increase chances of success • Development Planning: early SE to inform MDD • Risk Management: identify all technical risks and manage to closure — full and open discussion with PMs, PEOs, and SAE • Competitive prototyping: reduce risk for key technologies; increase competition • ECPs: work collaboratively with industry and stakeholders; mitigate requirements creep (non-Class 1 changes); use the CSB • Specs and standards in contracts: Reviews and Audits, Systems Requirements Document, Corrosion Prevention, ESOH, Reliability

5. Expectations of the SE Community • Robust, disciplined SE; sound engineering/technical management • Policy • Processes • People • Honest broker on performance/technical risks • Thorough understanding of programmatic technical issues • Competent insight into contractor technical planning and execution To meet these expectations, we must: (1) Institutionalize technical rigor and discipline across the life cycle of acquisition programs (2) Revitalize the AF SE workforce

6. Challenges • Planning: lack of sufficient technical input to make informed decisions • Little up-front and early technical mentoring, advice, and assessment • Program start decisions made with insufficient information — lack of concept/solution work prior to MDD • Programs pass milestones without adequate look at technical risk • Execution: technical issues/risks not discovered and addressed at the right time and at the right level • Beware of the “Sea of Green”— “I’m OK because I’ve got a plan” • Immature technologies and integration issues continue to drive cost and schedule impacts — we need to step up our game • ECPs are still rampant — impacts to AF enterprise unknown; must use CSBs more effectively • Maintain discipline of SE processes — don’t trade away rigor

7. AF SE Conference 3-5 Aug 2010 • THEME: “Systems Engineering: A Key Enabler to Development, Acquisition, & Sustainment” • Highlighted activities associated with improving the practice and management of SE across the Air Force • 224 attendees, principally representing the AF Engineering community • Product, Logistics, and Test Centers • Concept Development organizations (XR) • AFRL • HQ AFMC (Requirements, Engineering, Sustainment) • HAF/SAF • Invited presentations from DDR&E/SE • NDIA Reps

8. AF SE Conference3-5 Aug 2010 2 • Key takeaways • Robust SE contributes to successful programs • Implementing standard SE practices, processes, and policies fosters effective and efficient integrated life cycle management • Must take a balanced approach to reducing risk • Improved development and management of the technical workforce are critical to the future success of SE throughout the acquisition and sustainment life cycle • Accurate product and design data essential to maintain and improve operational safety, suitability, and effectiveness (OSS&E) – key to sustainment • Feedback was overwhelmingly positive • SAF/AQR, AF Center for Systems Engineering, and AFMC/EN have agreed to co-sponsor next year’s conference in Aug 11 at WPAFB

9. Process Focus • Areas of Strength • Development Planning/CCTDs: drives disciplined, repeatable concept definition; supports MDA at MDD • Air Force Program Support Review: standardized structure for specialty reviews & technical planning; includes PMO mentorship and assistance • Systems Engineering Assessment Model (SEAM): AFMC policy; identifies key SE processes & products • Common Tools: RI3, TDTS, LHA/ILA, … • Area for Improvement • Need to ensure new processes are fully implemented across our programs and adjustments made as necessary following implementation

10. Recapturing Technical Excellence – Two Initiatives Production and Deployment JCIDS 5000.02 MS A MS B MS C MS MS MS MS MS (-2) MS (-2) MS (-2) MS (-2) MS (-17 to-11) MS (-17 to-11) MS (-17 to-11) MS (-17 to-11) MS (-9 to -6) MS (-9 to -6) MS (-9 to -6) MS (-9 to -6) MS (-1) MS (-1) MS (-1) MS (-1) Technology Development Materiel Solution Analysis Joint Concepts ICD CDD CPD CBA Engineering and Manufacturing Development Strategic Guidance MDD MS (-18 to -12) MS (-18 to -12) MS (-18 to -12) MS (-18 to -12) Interim Feedback to PM Interim Feedback to PM Interim Feedback to PM Interim Feedback to PM Standardized OSD/AF reporting Guidance & Planning mtg Guidance & Planning mtg Guidance & Planning mtg Guidance & Planning mtg AFRB AFRB AFRB AFRB DAB DAB DAB DAB SEP Outcomes TRA OIPT OIPT OIPT OIPT Conduct DAPS assessment Conduct DAPS assessment Conduct DAPS assessment Conduct DAPS assessment Address preliminary findings Address preliminary findings Address preliminary findings Address preliminary findings TDS Identify CTEs Identify CTEs Identify CTEs Identify CTEs Assess CTEs’ TRLs Assess CTEs’ TRLs Assess CTEs’ TRLs Assess CTEs’ TRLs Statutory / regulatorydocument development, review, and approval Identify critical processes Identify critical processes Identify critical processes Identify critical processes Complete MRA Complete MRA Complete MRA Complete MRA Integrated risk assessment Review logistics readiness Review logistics readiness Review logistics readiness Review logistics readiness • AF Program Support Review Process (AF PSR) • SE Engagement up front and early with PMs to mentor/assist • AF Standard for Tech Review Process • Synchronized data requests to reduce program office & contractor impacts • Identification of technology maturation/integration risks • Collaborative with OSD to prevent surprises • Development Planning Process (DP) • S, T & E Engagement • Concept/Solution selection influence • SEP influence • TDS influence • T&E risk assessment • TRL data evaluation Prepare/review/complete PESHE documentation Prepare/review/complete PESHE documentation Prepare/review/complete PESHE documentation Prepare/review/complete PESHE documentation Prepare/review/complete technical documents (TDS, SEP, …) Prepare/review/complete technical documents (TDS, SEP, …) Prepare/review/complete technical documents (TDS, SEP, …) Prepare/review/complete technical documents (TDS, SEP, …) Conduct risk assessment Conduct risk assessment Conduct risk assessment Conduct risk assessment Monitor risk buy-down Monitor risk buy-down Monitor risk buy-down Monitor risk buy-down DP 2 DP 2 DP 2 DP 2 DP 1 DP 3 DP 3 DP 1 DP 3 DP 1 DP 1 DP 3

11. MDD Information Needs –Forthcoming DDR&E Policy • AF is assessing issues associated with industry engagement during DP / Early SE • Conflicts of interest, protection of proprietary information, solutions masquerading as concepts, contracting terms and conditions, numerous others ... ... To support those decisions, the DoD Components shall provide evidence at the MDD Review that will facilitate the determination that: a. The candidate materiel solution approaches have the potential to successfully address the capability gap(s), associated dependencies, and operational attributes. b. There exists a range of technically feasible solutions generated from across the entire solution space, as demonstrated through early prototypes, models, or data. c. Consideration has been given to near-term opportunities to provide a more rapid interim response to the capability need. d. The plan to staff and fund analytic, engineering, and programmatic activities supports the proposed milestone entry requirements. 11

12. Process • DP #1—Organize, train, equip • Mentor/advise program office • TRA: ID independent review team • Early risk identification • Gain program understanding • DP #2—Provide early status • Provide preliminary report to PM • TRA: approve CTEs • Address risks • Cross flow information • DP #3—Finalize products • Out brief findings to PM • TRA: approve TRLs • Capture open risks • Complete MS deliverables Post Review -Support AFRB/DAB -Conduct lessonslearned hot-wash MS MS (-2) MS (-17 to-11) MS (-9 to -6) MS (-1) Notional Timeline MS (-18 to -12) Interim Feedback to PM Guidance & Planning mtg AFRB DAB Leverage tech reviews DAPS (OSD-led) Technology Manufacturing Logistics ESOH Technical docs Integration Risk OIPT Conduct DAPS assessment Address preliminary findings Identify CTEs Assess CTEs’ TRLs Identify critical processes Complete MRA Review logistics readiness Review ESOH Prepare/review/complete technical documents (TDS, SEP, PESHE, …) Conduct risk assessment Monitor risk buy-down DP 2 DP 1 DP 3 Early technical engagement to seamlessly mentor/advise, independently assess & capture risks…so PMs can act * Statutory/regulatory process

13. Policy Focus • Areas of Strength • Acquisition Improvement Plan: acquisition workforce revitalization and improved requirements generation • Development Planning: focus of early systems engineering • Program Technical Assessments: collaborative, independent advice to senior decision makers • Standards revitalization: restoring standard practices • Areas for Improvement • Improve SE authority, independence, credibility/trust and visibility: Development Planning, AF Program Support Review processes right first steps • Reliability, Availability, and Maintainability: Guidance memo; explore potential use of standard tools, e.g., Risk Identification, Integration, and Illities (RI3) Guide

14. Recent Policy Changes • Reliability, Availability, and Maintainability (RAM) • Requires continuous reliability improvement, reliability growth plans • Guidance Memo published 20 Jul 10 • Program Technical Assessments • Establishes AF Program Support Review • Establishes responsibilities for Center-Level Technical Authority and PEO Chief Systems Engineers • Released over SAF/AQ signature, 10 Aug 10 • Development Planning (DP) Guide • Released over AFMC/A5 and AFSPC/A5 signatures, Jul10

15. Near-Term Policy Changes • Development Planning (changes to AFI 63-101 and AFI 63-1201) • Establishes Concept Characterization and Technical Description (CCTD) as early SE artifact to support Materiel Development Decision and Analysis of Alternatives • Adjudicated comments from Technical/Functional Coordination • Starting Mandatory Coordination Aug 10, publishing goal Oct 10 • CCTD Guide • In final review; expect release over AQR signature Aug 10 • Early Systems Engineering Guide • Content to be updated and folded into revision to DP Guide, start 1QFY11 • Corrosion Prevention & Control • F-22/F-35 issues driving increased emphasis • Evaluating adequacy of current policy and guidance

16. Specs and Standards • Acquisition program execution problems (technical, cost, and schedule) have a common root cause: • Blanket government removal of specs/standards (early 90s) • This led to • Absence of well-defined product characteristics • Inconsistent application of necessary SE processes • Specs and standards compliance becoming trade space in a highly cost-competitive environment • Plan for AF Acquisition Improvement requires … • Tailorable standard practices forSE processes (balanced) • Policy to mandate use in solicitations and contracts (TBD)

17. RAM Improvement • Include RAM considerations in Early SE (pre-Milestone A) activities • Incorporate Lessons Learned • Ensure realistic schedules and funding • Plan for longer operational use (extended service life) of systems and components • Utilize ManTech, CSE, and other resources • Increase training and awareness • Reconstitute AF PQM expertise • Increase emphasis on RAM in market surveys and source selections • Implement reliability growth programs

18. Personnel Focus • Areas of Strength • Revitalization of product center EN functional offices: centralizes SE career development and personnel management • Science, Technology, Engineering, & Mathematics (STEM): executing “Bright Horizons” strategic plan • Areas for Improvement • Acquisition Workforce Training: training capacity at DAU; in-process workforce growth • Identification of SE and DP resources across Air Force: functions and activities widely distributed

19. STEM – Revitalizing the AF Workforce AF STEM Definition The Air Force professionals having degrees in Science, Technology, Engineering, or Mathematics (STEM). Goal Areas: • Requirements/Inventory • Funding/Resources • Force Management • Continuum of Learning • Outreach • Evaluation/Analytics

20. Required Resources • Initial major resource required: TIME • Time to complete plans and processes • Time to implement plans and processes into policy; institutionalize • Time for the field and HQ to respond with higher fidelity resource requirements after policy implementation • Once numbers are in, need funding and billets to implement policy: PEOPLE • SE, DP, and SMEs manning • Supporting resources and personnel • Training sources; quality • OSD assistance to help enforce/implement the SE discipline desired through policy, processes, reviews, senior leadership message • SE is a key enabler of solutions to Acquisition priorities • Collaboration with all stakeholders -- industry, users, decision-makers -- is imperative to solve the problems

21. WHY IT’S IMPORTANT 21

22. BACKUP

23. Weapon System Acquisition Reform Act “The key to successful acquisition programs is getting things right from the start with sound systems engineering, cost estimating, and developmental testing early in the program cycle. The bill that we are introducing today will require the Department of Defense to take the steps needed to put major defense acquisition programs on a sound footing from the outset. If these changes are successfully implemented, they should help our acquisition programs avoid future cost overruns, schedule delays, and performance problems.” –Senator Carl Levin, Chairman, Senate Armed Services Committee “The Weapon System Acquisition Reform Act of 2009 is an important step in efforts to reform the defense acquisition process. This legislation is needed to focus acquisition and procurement on emphasizing systems engineering; more effective upfront planning and management of technology risk; and growing the acquisition workforce to meet program objectives.” –Senator John McCain, Ranking Member, Senate Armed Services Committee

24. SECDEF Guidance “Chief among institutional challenges facing the Department is acquisition.”

25. Recapture Acquisition Excellence • Acquisition Improvement Plan (AIP) • Revitalize the Air Force acquisition workforce • Improve requirements generation process • Instill budget and financial discipline • Improve Air Force major systems source selections • Establish clear lines of authority and accountability within acquisition organizations • Clear Priorities • Reinvigorate AF Nuclear Enterprise • Win Today’s Fight • Develop & Care for Airmen & Families • Modernizing Aging Air & Space Inventories • Recapture Acquisition Excellence Systems Engineering: Key enabler to recapture Acquisition Excellence

26. Air Force Systems Engineering Conference Tuesday, 3 August 2010 Wednesday, 4 August 2010 Thursday, 5 August 2010 116 KITCHEN 121 118 123 All briefings will be posted to the AF SE Conference CoP by 12 August https://rso.my.af.mil/afknprod/ASPs/Reg/ GroupAdmin.asp?Filter=23843&EventID=14647 &GroupID=19841 No-Host Social, 1700-1900 Hamlet Restaurant TION RECEP DESK 111 112

27. Session Descriptions – 3 August Session Descriptions – 4 August 1A: SE Policy (including JCIDS, DoDI 5000.02, and Early SE). Rooms 118/121/123, 1330-1500. Moderators: Mr. Bob Reifenberg, ASC/ENS, and Mr. Jeff Loren, SAF/AQRE. This session will address recent and coming policy changes that impact Early SE, including WSARA and its impacts to DoDI 5000.02; requirements community interfaces; governance; resources; the Early SE “V” and CCTD; support for AoAs; and integrated Development Plans/roadmaps. 1B: Revitalization of Specs and Standards. Rooms 111/112, 1330-1500. Moderator: Mr. Scott Kuhnen, AFMC/ENS. Presentations will address current AF policy initiatives and on-going efforts at the MAJCOM/AF/DoD levels to revitalize specifications and standards. 1C: Systems Engineering Research. Room 116, 1330-1500. Moderator: Dr. David Jacques, AFIT/ENV. This session will discuss the unique nature of SE research, new directions being pursued, and potential benefits for the Department of Defense. Three leading researchers will provide their perspectives and examples of DoD-related SE research. 1D: Modeling & Simulation. Room 118, 1530-1700. Moderator: Lt Col John Simeoni.M&S is a valuable tool that can support and enhance the acquisition process. M&S helps identify key ops and design factors early and supports rapid design, fabrication, and testing. Targeted application of M&S has the potential to reduce risk, shorten timelines, and provide data for making sound managerial and technical decisions. The AF must plan for the use of M&S early in the life cycle as an integral part of the SE process. 1E: Architectural Frameworks. Rooms 121/123, 1530-1700. Moderator: Dr. John Colombi, AFIT/ENV. Architecture has been long used as a tool to understand and communicate complexity in order to enable structured decision-making. This session will focus on ways to make AF architecture more useful and relevant. Presentations will provide attendees with new ideas, policy, and perspectives on architecting to better support acquisition and pre-acquisition activities. 1F: Human Systems Integration (HSI). Rooms 111-112, 1530-1700. Moderator: Col Larry Kimm, AFHSIO. Systems engineers will learn how HSI is a life cycle process and applies from before MS A (supporting development planning and early systems engineering) through operations and sustainment, with information on available guidance, tools and resources. 2A: Integrated Master Plan (IMP) & Integrated Master Schedule (IMS). Rooms 118/121/123, 1330-1500. Moderator: Mr. Sam Johns, AF PM&AE.This session is targeted toward increased understanding of the importance of IMP & IMS and the associated role of the Systems Engineer. It will cover current Air Force IMP/IMS policy; engineering's role in IMP development and construction; an IMP/IMS integration tutorial; and the IMP/IMS in the Integrated Baseline Review process. 2B: Reliability. Rooms 111/112, 1330-1500. Moderator: Mr. Sam Brown, SAF/AQRE. Presentations will address ongoing OSD efforts to improve reliability guidance and implementation across the Services, and challenges being faced by Air Force programs as they implement reliability programs. Attendees will be able to share their thoughts and ideas following the presentations. 2C: Environment, Safety and Occupational Health (ESOH). Room 116, 1330-1500. Moderator: Mr. Ken Dormer, SAF/AQRE. This session will address the integration of ESOH design and technical risk management considerations across the life cycle as a part of a disciplined systems engineering approach. Speakers will discuss OSD initiatives to improve ESOH-engineering integration, the implementation perspective of one AF Center, and DoD efforts to update MIL-STD-882D. 2D: Test and Evaluation (T&E). Rooms 121/123, 1530-1700. Moderator: Dr. Michael Deis, HQ AFMC/A3. This session will provide perspectives on the implementation of WSARA systems engineering and developmental testing requirements. A question and answer period will follow. 2E: Life Cycle Supportability. Room 118, 1530-1700. Moderator: Mr. Edward Durell, SAF/AQRE. Session will cover "Logistician Expectations of Systems Engineering" as well as "Supportability Perspectives from Opposite Ends of the Weapon System Life Cycle." A panel discussion on life cycle supportability will follow presentations. 2F: Cool Tools (ARM, SEAM, RI3, RTT etc). Rooms 111/112, 1530-1700. Moderator: Ms. Karen Bausman, AFIT/SY. This session will include two presentations focused on enlightening the workforce to the strategic planning activities for SE tools. Presentations will address the overarching AF plan for SE tools, as well as the use of SE tools throughout AFMC. Q&A will follow with a focus on coordination among different tools efforts. Air Force Systems Engineering Conference 3-5 August 2010 Systems Engineering: A Key Enabler to Development, Acquisition, & Sustainment SAFTAS Conference Center1550 Crystal DriveArlington, Virginia