Critical Success Factors in Systems of Systems Engineering

1. Critical Success Factors in Systems of Systems Engineering Arthur Pyster Senior VP and Director of Systems Engineering and Integration, SAIC Pat Gardner Chief Technology Officer, Tactical Systems & Solutions Business Unit October 25, 2006

2. Objectives • Clarify distinction between a system and a system of systems (SOS) • Discuss lessons learned from hands-on systems of systems engineering (SOSE) practice • Identify and explain success factors • Warn against some common remedies that do not work © SAIC, 2006

3. Many Definitions of SOS “…a set or arrangement of interdependent systems that are related or connected to provide a given capability. The loss of any part of the system will significantly degrade the performance or capabilities of the whole. The development of a SOS solution will involve trade space between the systems as well as within an individual system performance. An example of a SOS would be a combat aircraft. While the aircraft may be developed as a single system, it could incorporate subsystems developed for other aircraft. For example, the radar from an existing aircraft may be incorporated into the one being developed rather than developing a new radar. The SOS in this case would be the airframe, engines, radar, avionics, etc. that make up the entire combat aircraft capability.” [Joint Capability and Integration Development System, 2005] © SAIC, 2006

4. Many Definitions of SOS (continued) “… an integrated force package of interoperable systems acting as a single system to achieve a mission capability. Typical characteristics include a high degree of collaboration and coordination, flexible addition or removal of component systems, and a net-centric architecture…” [Naval “Systems of Systems” Systems Engineering Guidebook, 2005] “… a set or arrangement of systems that results when independent and useful systems are integrated into a larger system that delivers unique capabilities…” [Systems of Systems Engineering Guide (October 6, 2006 draft)] “Systems of systems exist when there is a presence of a majority of the following five characteristics: operational and managerial independence, geographical distribution, emergent behavior, and evolutionary development.” [Sage and Cuppan 2001] © SAIC, 2006

5. U.S. National Airspace System as an SOS • Air space management and safety achieved by all component systems working together • Individual systems provide useful services such as navigation to a pilot • Individual systems are acquired independently with different contractors • Individual systems come and go routinely • Individual systems are operated by FAA, airports, airlines, NOAA, … • Highly network centric • Standard protocols and interfaces abound • Geographically dispersed • Overarching CONOPS and architecture provide for evolutionary development • Despite extensive modeling, system complexity leads to emergent behavior • Extensive coordination is central to achieving high levels of efficiency and safety © SAIC, 2006

6. Examples • The Boeing 777 is not an SOS. It is a large complex product line system. • The U.S. National Airspace System, which includes fleets of Boeing 777s, is an SOS. • A single UAV is not an SOS. It is a member of a product line of complex vehicles. • FCS, which includes UAVs, is an SOS. • Neither the NAS, nor any civilian aviation system, was designed anticipating UAVs, but they are now being added. Individual system types come and go. Individual system instances come and go. • System boundaries are often flexible and fuzzy – not what we like in systems engineering • Emergent behavior – 9/11 changed the CONOPS for the NAS and how we handle flight security. Example evolutionary implication: primary radars were on the way out. No longer. © SAIC, 2006

7. Systems of Systems Engineering Army USAF FAA SOS Systems Engineering designs and integrates programs in which: • The SOS has Concepts of Operation that are a superset of the concepts of the individual member systems • The behavior of the SOS is characterized by • A high degree of functional collaboration among the member systems • A focus on information dominance that cannot be achieved by a single system acting alone (e.g., through the use of enterprise service-oriented architectures) • Member systems may enter or leave (and return) while SOS operations persist • SOS acquisition has objectives that cannot be achieved by individual procurements • A family of related systems with interlocking requirements and missions • Commonality and modularity objectives for design • SOS managed risk and design-to-cost for a group of major systems • The first-tier SOS subcontractors are prime contractors for major systems • Integrated End Items may have complete uses outside the System of Systems • End Items have unique requirements that do not apply to the System of Systems as a whole NII/DISA Navy USAF JFCOM FAA NASA UK Army Acquisition FAA All © SAIC, 2006

8. SAIC FCS SOSE Experience • (SAIC Prime) SAIC developed and exercised the SOSE methodology for FCS • SOSE “Double Vee” process to engineer the system • SOS architecture for network centric warfare • (LSI with Boeing) SAIC engineers lead the Architecture Development team • Full Enterprise and HW/SW layered architecture • NII and J6 vetted and approved methodology • (LSI) SAIC engineers provide SE&I management leadership • Managing Prime Item engineering and development • Leading vehicle/platform integration • (LSI) SAIC engineers lead the Integrated Simulation and Test team • Distributed HW/SW-in-the-loop simulation and System Integration Laboratory environment • Direct linkages: analysis – simulation – SIL – full scale test © SAIC, 2006

9. SOSE Process Overview This is a particular process example developed for an SOS pursuit in Europe. It applies well to many classes of problems in SOS engineering. Prime Contractors Many different “SOSE process” descriptions are possible. All are adaptations of the SE “Vee” model. © SAIC, 2006

10. Team Organization is Crucial • Coordinating multiple SE teams through documentation, reporting, and ICD’s does not work effectively • One systems engineering team • Integrate a distributed team using Top Down SE Management, integrated tasking, and clear accountability • Use common networked tools and processes that ensure communication of information and rapid issue resolution • Team must deal with all levels of SE issues from the End Item through subsystems (layered architecture) • Integrate and train the SE Team • Train to modern process standards – legacy won’t work for modern, software-intensive SOS that evolve over time • Must use up-to-date software architectures and tools • System Engineering Management is the integrating factor • Synchronize the team • Integrate the team vertically and horizontally © SAIC, 2006

11. Communications • Canons of systems engineering • Make sure everyone knows what the problem is • Make sure everyone is solving the same problem • When something changes, make sure everyone knows about it • In a complex, layered SOS, “design to spec and wait to verify” is insufficient • Critical changes and design trades will be made after specifications have been cut and contractors selected • Proactive communications are required on a frequent basis • Within the integration team • Between the team and the stakeholders • Among the execution team (integrators and suppliers) • Use formal methods; e.g., reviews, and technical interchange meetings • Use innovative methods Managing communications is the job of the System of Systems Lead Integrator © SAIC, 2006

12. Standards Compliance is an Asset • Proprietary, outdated, noncompliant processes ultimately compromise team integration • Challenge: organizations with different (or nonexistent) processes • System engineering standards handle modern, complex problems • ISO-15288 – Product life cycle synchronization • ANSI/IEC-632 – System Engineering and the Enterprise • IEEE-1471 – Architecture development using multiple frameworks (eg. Tailored for multimission SOS, HWCI, CSCI, information systems) • Incorporate DODAF, FEAF, Zachman, and SW frameworks • Standards lead to predictable outcomes that reduce risk • Compliant processes integrate team operations • Common expectations of outcomes and validation standards • Compliance requires vertically integrated SE management, diligence, and verification • Core responsibility of the SOS integrator © SAIC, 2006

13. Continuous Verification and Validation System of Systems Life Cycle The System of Systems life cycle is different from (and encapsulates) the life cycles of the development End Items Control the SOS development with a structured set of control gates recorded in SOS IMP/IMS Manage the System of Systems deployment by synchronizing the life cycle events of the constituent systems © SAIC, 2006

14. SOSE Management Tasks The System of Systems Integrator specifies and controls the life cycle support systems Requirements and Specifications Plans and Constraints Prime Contractors for System Developments Designs and Progress Data Schedules and Events © SAIC, 2006

15. Role of Simulation in SOS Engineering System of Systems design and integration are complex activities that can be effectively supported by continuous V&V using Modeling and Simulation tools. © SAIC, 2006

16. SOS Have Layered Architectures • Requirements Management, Configuration Management, and Change Management quickly lose synchronization without a vertically integrating top-down mechanism • System of Systems have a layered architecture that is the organizing principle for system engineering management • System breakdown structure for platforms • Architectures and flowdown functional analysis • Requirements derivation and flowdown • Mission and End Item integration • Manage configuration complexity by Architecture Level • Explicit links to functions, components and interfaces on own level (modeled in the architecture) – horizontal integration • Explicit links “up” to functions, components and interfaces on the next level – derivation and realization are explicit – vertical integration Effective SOS management control depends critically on organizing the complex elements of the program. © SAIC, 2006

17. SOSE management control using Architecture Levels applies to core system engineering tasks: Work Breakdown Structure Product Breakdown Structure Commonality and modularity design Specification development Architecture development Requirements management Configuration management Technology assessment and insertion Horizontal Integration Horizontal Integration Horizontal Integration SOS Layered Architecture Structure Stakeholders System of Systems End Items Vertical Integration (each End Item) Common Systems Mission Systems, Subsystems and Components Control the SOS structure using Architecture Levels © SAIC, 2006

18. Checklist for SOSE Success • Do detailed planning and analysis based on knowledge of the structure of the problem space (and adherence to the plan) • Continually validate the SOS CONOPS – it is guaranteed to be wrong, keep making it better • Continually model, simulate, and apply other techniques at the SOS level to understand complex behaviors, emergent behaviors, requirements tradeoffs, and the impact of change • Concentrate on horizontal integration and synchronization – 80% of the time design changes and decisions don’t matter, but the other 20% will kill you • Execute in a disciplined manner that identifies, measures, and prevents problems • Design your organization to promote communications and integrated execution – use collaboration tools to help • Train and execute to recognized best practices and standards © SAIC, 2006

19. Finally • Systems of system engineering (SOSE) is not fundamentally different from the systems engineering of large complex systems • The most complex (single) systems have emergent behavior, require extensive and continuous modeling and simulation to be understood, have evolving requirements and implementation, many subsystems with elaborate communications, … • Nevertheless, the scale of the largest SOS is daunting and challenges our processes, techniques, and tools © SAIC, 2006