Episode 3 Validation Strategy

1. Episode 3 Validation Strategy Episode 3 - CAATS II Final Dissemination Event Alistair Jackson & Patrick Lelièvre EUROCONTROL & Airbus Episode 3 Brussels, 13 & 14 Oct 2009

2. Create Validation Strategy • On the basis of the following inputs: • The project objectives, timescale and resources • The target operational concept and its current maturity • The main expectations of the key internal and external stakeholders • Establish as outputs: • The project validation objectives and targets • The means by which these can be practically achieved Episode 3 - CAATS II Final Dissemination Event Key function in establishing the principles of the shared working methods, practices and assumptions necessary to achieve an integrated approach to validation

3. Initial Challenges for Episode 3 Validation • Aligning with SESAR Concept • Complexity and scale of concept • Performance orientation • Existence of a DOW prior to establishment of the strategy • Using the E-OCVM • Achieving the necessary degree of understanding throughout the project (absence of appropriate training at the time) • Adapting E-OCVM to a distributed, multi-partner project • Addressing a concept with many elements and contexts of application • Lack of precedents for integrated, performance based validation • Availability of concept description (timing and level of detail) Episode 3 - CAATS II Final Dissemination Event

4. EP3 Project Objectives • Reminder of Episode 3 Project Objectives • Provide detail on key concept elements in SESAR • Initial operability and performance assessment • Initial technical impact assessment • Analysis of available tools for SESAR validation • Report on validation methodology Episode 3 - CAATS II Final Dissemination Event

5. EP3 Validation Objectives • Validation targets at project Level: • Establish a process for clarifying and constructing a shared understanding of the concept • A common performance framework for performance related validation • Understanding of validation processes and E-OCVM • Good quality validation exercise designs • Identification and development of early stage (V1, V2) validation techniques • Identify requirements for a credible, systematic validation process as input to future SESAR validation Episode 3 - CAATS II Final Dissemination Event

6. V1 V1 V1 V2 V2 V2 V3 V3 V3 Increasing Level of Concept Maturity Increasing Level of Concept Maturity EPISODE 3 EPISODE 3 WP2 WP2 Overall Overall Updated Updated Updated Updated Target Target Validation Validation Overall Overall Overall Overall Concept Concept Strategy Strategy Validation Validation Validation Validation Strategy Strategy Strategy Strategy WP6 WP6 WP6 WP5 WP5 WP4 WP4 Operational Operational WP3 WP3 Updated Updated Updated Updated Context Context Operational Operational Operational Operational Operational Operational Concept Level Concept Level Context Context Context Context Context Context Validation Validation Validation Validation Validation Validation Strategy Strategy Strategy Strategy Strategy Strategy OIs OIs Validation Validation Exercises Exercises Validation at Project and WP level Adopted a two-tier approach to validation with overall Project validation targets and WP (domain) validation strategies (c.f. SESAR DP 4.2.1) Episode 3 - CAATS II Final Dissemination Event Episode 3 - CAATS II Final Dissemination Event

7. Coupling operational & technical validation • Enablers maturity based on TRL process • To make available mature competitive technology and solutions for Industry, at the “right” time Episode 3 - CAATS II Final Dissemination Event • Concept maturity based on E-OCVM • The iterative process by which thefitness-for-purpose of a new system or operationalconcept being developed is established

8. What we did in EP3 Validation 1 • Created guidelines (and templates) to communicate some key validation principles and to adapt the E-OCVM to EP3 • Used the review of exercise plans (and reports) to both monitor and coach on validation exercise quality • Explored the use of validation techniques adapted to the early stages of the validation lifecycle – Expert Groups, Gaming, Influence Diagram Modelling • Developed Concept clarification process (DODs, etc.) • Expanded on SESAR Performance Framework Episode 3 - CAATS II Final Dissemination Event

9. What we did in EP3 Validation 2 • Contributed to improvement of the ‘Environmental’ and ‘Safety Cases’ • Analysed how to achieve integrated, performance based, validation • Constructed an understanding of the difficulties of validation amongst the partners and generated important feedback for E-OCVM • Matured ATM Capability Level 2 Aircraft and ATC System Enablers (Initial 4D and ASAS S&M) up to TRL 4 (First step from E-OCVM V2 to V3) • Enhanced industry validation methods  Integrated air-ground industrial platform, performance evaluation tool Episode 3 - CAATS II Final Dissemination Event

10. What we did NOT do ‘Open Issues’ • Perform systematic top-down validation and consequently: • Choose exercises on the basis of validation priorities • Provide a systematic cover of the concept • Achieve integrated performance based validation • Give sufficient priority to establishing an effective, standardised, approach to maturity assessment • Establish an effective coupling between the operational concept validation and the technical verification processes • Interdependencies (deliverables and schedule) between E-OCVM and TRL still to be defined (ops concept, procedures, standardisation needs, system prototypes, equipments, lab & flight tests…) • Address all the transversal areas effectively – e.g. no human performance case, no business case, no integrated approach Episode 3 - CAATS II Final Dissemination Event

11. Key Messages for SESAR • Importance of initial validation strategy at early stages in establishing shared working practices, references and understanding • The need for a work breakdown structure reflecting validation priorities • Shared maturity assessment process – ‘maintain a common maturity map of concept elements’ to prioritise validation activity • Make explicit, document and share assumptions about key aspects of the validation process (concept, operational environment, traffic, etc.) • The importance of coupling the operational and technical validation processes Episode 3 - CAATS II Final Dissemination Event