Dr. Dinesh Verma Associate Dean and Professor Stevens Institute of Technology, Hoboken, New Jersey

1. Dr. Dinesh Verma Associate Dean and Professor Stevens Institute of Technology, Hoboken, New Jersey Mr. Cliff Geiger President Unified Industries, Springfield, Virginia Mr. Louis Kratz ADUSD, Logistics Programs and Plans Office of Secretary of Defense, Pentagon System Operational Effectiveness:Assessment of “Cause-And-Effect” Dependency between System Design & Support Systems and Supportability Engineering Symposium October, Dallas, Texas

2. Why? What? How? Presentation Outline • The System Integration Business Model andPerformance Based Contracting • Suggested Definition of an Approach: System Operational Effectiveness Framework • Suggested Methods and Metrics:Trade-offs Between Performance and Cost; Dependency between Design and Support

3. Systems Integration • Definition • From Webster's Revised Unabridged Dictionary (1913) (web1913) • Integration \In`te*gra”tion\, n.[L. integratio a renewing, restoring: cf. F. int[`e]gration.] 1. The act or process of making whole or entire. • Definition • Systems Integration Is an “Entire View” of Missions and Operational Environments Achieved By Combining Capabilities of Platforms, Systems, Operators, & Support to Maximize Performance • Role of a System Integrator: Evolves from Providing Products, Systems & System Elements… to Providing Functionality or Solutions • Role of a Customer: Evolves into the concept of Performance Based Contracting - from Contracting for Products, Systems, & System Elements… to Contracting for a Functionality or a Capability

4. Systems Integration • The System Integrator • Manages System Effectiveness Through: • System Performance - Guaranteed End-To-End Performance • Total Ownership Cost (TOC) - Guaranteed Affordability • Assumes Cost, Schedule, and Performance Risks Advanced Applications Desired Functionality within the Cost (Life Cycle Cost) and Schedule Envelope Development Capabilities Subcontractors and Vendors System Integrator Manages Performance/ TOC; Assumes Risk Customer Long Term Support Management Expertise Interface Management

5. Systems Integration • The System Integrator • Manages System Effectiveness Through: • System Performance - Guaranteed End-To-End Performance • Total Ownership Cost (TOC) - Guaranteed Affordability • Assumes Cost, Schedule, and Performance Risks Advanced Applications Development Capabilities Subcontractors and Vendors System Integrator Manages Performance/ TOC; Assumes Risk Customer Performance Based Contracting; System Concept of Operations Long Term Support Management Expertise Interface Management

6. Systems Engineering - A Definition ...translation of a need/deficiency into a system architecture through the iterative process of functional analysis, allocation, implementation, optimization, test, and evaluation; ...incorporation of technical parameters to assure compatibility between physical & functional interfaces, hardware & software interfaces, in a manner that optimizes system definition and design; and ...integration of performance, manufacturing, reliability, maintainability, supportability, global flexibility, scaleability, upgradeability and other specialties into the overall engineering effort. Requirements Ops Analysis Functional Analysis Function/Cost Allocation Architecture Modeling and Trade-Off Alternatives Design Build/Test Deploy Commercial Market Analysis Cost Analysis Establish Cost Goals Validate Cost Goals Monitor Achievement Manpower/ Personnel Modularity Standards Commonality RMA Training Documentation Maintenance Test/Support Facilities Supply Support PHS&T

7. Support/Logistics Products Development Design Influence Lifetime Support Evaluation Tech. Refreshment Fleet Feedback Elements of Logistics Support: • Supply Support (Spare/Repair Parts) • Maintenance Planning • Test/Support Equipment • Technical Documentation/IETM • Manpower/Personnel • Training/CBT • Facilities; PHS&T • Design Interface; Computing Support Requirements For The: Systems and Supportability Engineering Concept of Operations Design Reviews and Evaluation TLR & Maintenance Concept Maintainability Analysis Level of Repair Maintainability Prediction Functional Analysis Functional Flow/ Data Flow Diagrams Allocation of System Requirements System Architecture/ Selection of System Elements Have Requirements Been Met? Support Test/ Evaluation System Test & Evaluation (Hot Bed Testing) Failure Mode, Effects, and Criticality Analysis (FMECA) Reliability Centered Maintenance (RCM) Sustaining SYSTEM Support Yes Maintenance Task Analysis (MTA) Detailed Support Product List Fault Tree Analysis (FTA) No System Reliability Analysis, Modeling, and Allocation Reliability Prediction System Redesign/ Improvement Cost as an Independent Variable (CAIV): Design to Affordability Analysis (Strategic Decision Making) Technology/Standards Evolution and COTS Products Market Surveillance Technology Refreshment

8. Reliability/ Design “Cause” Operation Operational “Effect” Supportability/ Maintainability/ Logistics Maintenance Requirements Performance Functions Priorities Technical Reliability Effectiveness Availability Maintainability System Supportability Effectiveness Operation Operational Process Maintenance Effectiveness Efficiency Logistics System Life-Cycle Cost/Design to Affordability System Operational Effectiveness System Uptime System Downtime Time to Failure (TTF) Time to Support (TTS) Time to Maintain (TTM)

9. System Operational Effectiveness

10. 75% COTS 5:1 Reduction in Cost 100X Increase in Throughput Application Software Unique Software Application Software Standard Application Program Interface Results in New System Partitioning Unique Hardware Unique Sensor & Weapon Interfaces COTS Based Hardware and Software Computing Infrastructure System Architectural Evolution Past Present Future • Extend Open Standards to Focus on Interoperability (Net-Centric) • Utilize High Throughput of Commercial Technologies to Address Automation and Supportability • “Smart” Software • Unique Hardware and Software for Computing Infrastructure • Closed/Proprietary Architecture/ Interfaces • Mission Capabilities Created By the Combination of Unique Software and Hardware • Commercially Available Technologies for Computing Infrastructure • Open Architecture Using Commercial Standards for Interfaces • Mission Capabilities Created By Reliable, Dependable, Durable Software Applications Tightly Linked Proprietary Interface

11. ILS Products Development Design Influence Lifetime Support Reliability Maintainability Supportability • Redundancy • Reconfigurability • De-Rating • System Criticality Assessment • Single Points of Failure • Degraded Modes of Operation • Metrics • Tools Evaluation Tech. Refreshment Fleet Feedback Elements of Logistics Support: • Supply Support (Spare/Repair Parts) • Maintenance Planning • Test/Support Equipment • Technical Documentation/IETM • Manpower/Personnel • Training/CBT • Facilities; PHS&T • Design Interface; Computing Support Requirements For The: Systems and Supportability Engineering Concept of Operations Design Reviews and Evaluation TLR & Maintenance Concept Maintainability Analysis Level of Repair Maintainability Prediction Functional Analysis Functional Flow/ Data Flow Diagrams Allocation of System Requirements System Architecture/ Selection of System Elements Have Requirements Been Met? Support Test/ Evaluation System Test & Evaluation (Hot Bed Testing) Failure Mode, Effects, and Criticality Analysis (FMECA) Reliability Centered Maintenance (RCM) Sustaining SYSTEM Support Yes Maintenance Task Analysis (MTA) Detailed Support Product List Fault Tree Analysis (FTA) No System Reliability Analysis, Modeling, and Allocation Reliability Prediction System Redesign/ Improvement Cost as an Independent Variable (CAIV): Design to Affordability Analysis (Strategic Decision Making) Technology/Standards Evolution and COTS Products Market Surveillance Technology Refreshment

12. ILS Products Development Design Influence Lifetime Support Evaluation Tech. Refreshment Fleet Feedback Elements of Logistics Support: • Supply Support (Spare/Repair Parts) • Maintenance Planning • Test/Support Equipment • Technical Documentation/IETM • Manpower/Personnel • Training/CBT • Facilities; PHS&T • Design Interface; Computing Support Requirements For The: Systems and Supportability Engineering Concept of Operations Design Reviews and Evaluation Reliability Maintainability Supportability TLR & Maintenance Concept Maintainability Analysis Level of Repair Maintainability Prediction • Maintenance Concept • Accessibility • Performance Monitoring and Fault Localization • Built-In Test Coverage • System Modularity/De-Coupling • Condition and Usage Monitoring • Metrics • Tools Functional Analysis Functional Flow/ Data Flow Diagrams Allocation of System Requirements System Architecture/ Selection of System Elements Have Requirements Been Met? Support Test/ Evaluation System Test & Evaluation (Hot Bed Testing) Failure Mode, Effects, and Criticality Analysis (FMECA) Reliability Centered Maintenance (RCM) Sustaining SYSTEM Support Yes Maintenance Task Analysis (MTA) Detailed Support Product List Fault Tree Analysis (FTA) No System Reliability Analysis, Modeling, and Allocation Reliability Prediction System Redesign/ Improvement Cost as an Independent Variable (CAIV): Design to Affordability Analysis (Strategic Decision Making) Technology/Standards Evolution and COTS Products Market Surveillance Technology Refreshment

13. ILS Products Development Design Influence Lifetime Support Evaluation Tech. Refreshment Fleet Feedback Elements of Logistics Support: • Supply Support (Spare/Repair Parts) • Maintenance Planning • Test/Support Equipment • Technical Documentation/IETM • Manpower/Personnel • Training/CBT • Facilities; PHS&T • Design Interface; Computing Support Requirements For The: Systems and Supportability Engineering Concept of Operations Design Reviews and Evaluation Reliability Maintainability Supportability TLR & Maintenance Concept Maintainability Analysis Level of Repair Maintainability Prediction Functional Analysis Functional Flow/ Data Flow Diagrams Allocation of System Requirements System Architecture/ Selection of System Elements • System Commonality • Physical Commonality • Operational Commonality/HMI Standardization • Functional Commonality • Standard Parts • Standard Tools/Equipment • Intuitive User Interface • COTS/GOTS Selection and Assessment • Open/Popular System Standards Compliance • Multiple Vendors • Technology Maturity • Metrics • Tools Have Requirements Been Met? Support Test/ Evaluation System Test & Evaluation (Hot Bed Testing) Failure Mode, Effects, and Criticality Analysis (FMECA) Reliability Centered Maintenance (RCM) Sustaining SYSTEM Support Yes Maintenance Task Analysis (MTA) Detailed Support Product List Fault Tree Analysis (FTA) No System Reliability Analysis, Modeling, and Allocation Reliability Prediction System Redesign/ Improvement Cost as an Independent Variable (CAIV): Design to Affordability Analysis (Strategic Decision Making) Technology/Standards Evolution and COTS Products Market Surveillance Technology Refreshment

14. System Architecture “Goodness” Commonality Modularity Standards Based RMT • Physical Modularity • Ease of system element upgrade • Lines of modified code • Amount of labour hours for system rework • Ease of operating system upgrade • Lines of modified code • Amount of labour hours for system rework • Functional Modularity • Ease of adding new functionality • Lines of modified code • Amount of labour hours for system rework • Ease of upgrade existing functionality • Lines of modified code • Amount of labour hours for system rework • Orthogonality • Are functional requirements fragmented across multiple processing elements and interfaces? • Are there throughput requirements across interfaces? • Are common specifications identified? • Abstraction • Does the system architecture provide and option for information hiding? • Interfaces • # of Unique Interfaces per System Element • # of Different Networking Protocols • Explicit versus Implicit Interfaces • Does the architecture involve implicit interfaces? • # of Cables in the System • Open Systems Orientation • Interface Standards • # of Interface Standards/# of Interfaces • Multiple Vendors (Greater than 5) Exist for Products Based on Standards • Multiple Business Domains Apply/Use Standard (Aerospace, Medical, Telecommunications) • Standard Maturity • Hardware Standards • # of Form Factors/# of LRUs • Multiple Vendors (Greater than 5) Exist for Products Based on Standards • Multiple Business Domains Apply/Use Standard (Aerospace, Medical, Telecommunications) • Standard Maturity • Software Standards • # of proprietary & unique operating systems • # of non-std databases • # of proprietary middle-ware • # of non-std languages • Consistency Orientation • Common Guidelines for Implementing Diagnostics and Performance Monitoring and Fault Localisation • Common Guidelines for Implementing OMI • Reliability • Fault Tolerance • % of mission critical functions with single points of failure • % of safety critical functions with single points of failure • Critical Points of Delicateness (System Loading) • % Processor Loading • % Memory Loading • How critical is this? • % Network Loading • How critical is this? • Maintainability • Expected MTTR • Maximum Fault Group Size • Is system operational under maintenance? • Accessibility • Are there space restrictions? • Are there special tool requirements? • Are there special skills requirements? • Testability • # of LRUs covered by BIT (BIT Coverage) • Reproducibility of Errors • Logging/Recording Capability • Create system state at time of system failure? • Online Testing • Is system operational during external testing? • Ease of access to external testpoints? • Automated Input/Stimulation Insertion • Physical Commonality (Within the system) • HW Commonality • Number of Unique LRUs • Number of Unique Fasteners • Number of Unique Cables • Number of Unique Standards Implemented • SW Commonality • Number of Unique SW Packages Implemented • Number of Languages • Number of Compilers • Average Number of SW Instantiations • Number of Unique Standards Implemented • Physical Familiarity (From other systems) • % Vendors Known • % Subcontractors Known • % HW Technology Known • % SW Technology Known • Operational Commonality • % of Operational Functions Automated • Number of Unique Skill Codes Required • Estimated Operational Training Time - Initial • Estimated Operational Training Time - Refresh from Previous System • Estimated Maintenance Training Time - Initial • Estimated Maintenance Training Time - Refresh from Previous System

15. A Non-Response from Academia • In a 1990 Survey (Conducted by Virginia Tech): • 73 programs (Universities) were identified as offering a degree in Systems Engineering, however a majority of these programs are aligned with a functional area (Industrial, Information, Mechanical) • Only 8 programs appeared to offer an option for interdisciplinary Systems Engineering • Of these 8, a subset focused on System Design but none offered a focus on System Reliability, Maintainability, and Supportability • Only 2 of these options had electives on subjects related to logistics • In a 1998 Survey (Conducted by University of Virginia): • 23 programs offering Systems Engineering degrees were identified, however a majority of these programs are traditional industrial engineering programs • Only 7 programs focused on a system analysis and design view

16. Conclusions • The desire and the lexicon has supported integrating support and logistics issues in the systems engineering process for decades. • The current emphasis on COTS-based architectures, system affordability, and network centric warfare mandates it! • Accordingly it is essential to clear the haze surrounding the domain and scope of supportability and logistics • This is essential to translating the rhetoric related to CAIV and Performance Based Contracting into reality • Understanding the domain and scope of supportability and logistics facilitates leveraging the intuitive “cause and effect” relationship between system design and system support… and ultimately system affordability. Production & Deployment Utilization & Phase-out Conceptual & Preliminary Development Detailed Engineering/ Development/T&E Production Infrastructure Design and Development Production Operations Logistics & Support Infrastructure Design and Development Support & Maintenance Operations