Peter Stastny Head of Safety Regulation Unit EUROCONTROL peter.stastny@eurocontrol.int ICAO Montreal, Thursday 29 March

1. Peter StastnyHead of Safety Regulation UnitEUROCONTROLpeter.stastny@eurocontrol.intICAOMontreal, Thursday 29 March 2007 Ensuring the Safety of Future Developments

2. Overview A Performance-based Approach to Safety • Monitoring Safety Performance • Performance Measurement – the essential tool • Measuring Safety Maturity • Performance-based Approach to Managing Risk • Risk management – part of SMS • Safety oversight aspects – the role of ESARR 1 • Risk classification methodology – defining tolerable safety • Conclusions

3. Reactive Proactive Performance – part of SMS Risk Assessment New Systems Recruitment/ Selection Procedures Operational Processes Incident Reporting Safety Surveys and Follow-up Risk Assessment ATM Procedures Training Incident Investigation Interface ATS CNS, AIM, Airports Competency Checks Lessons Learnt CNS /AIM Maintenance Procedures Risk Assessment Airspace Changes Refresher/ Advanced Training “Historic” Safety Performance Measurement Risk Assessment Software Changes Emergency Procedures Safety Maturity Measurement A systematic approach to the management of safety

4. IN THE FUTURE . . . EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL SAFETY REGULATORY REQUIREMENT (ESARR) ESARR 1 SAFETY OVERSIGHT IN ATM EDITION : 1.0 EDITION DATE : 27-02-2002 STATUS : RELEASED ISSUE CLASS : GENERAL PUBLIC Performance – part of Safety Oversight ESARR 1…. • Defines minimum arrangements/ processes for ATM safety oversight: • with certification • or without certification • A unique basis for harmonising and reinforcing the role and operation of national regulatory bodies • Requires monitoring of safety performance as part of safety oversight

5. Key Principles Interested Parties • Information to public/stakeholders • Call to action by stakeholders Public/ Industry KPIs States/ Industry Performance Indicators • Facilitates identification of scope of action required • Facilitates management of improvement of service Management Measurement Metrics Organisational Level (Service Providers) Laws Incidents Audit Compliance Information Culture Accidents Resources SMS Procedures AIB Recommendations The whole process needs to be a continuous improvement activity Safety Performance Measurement

6. Occurrence-based performance measurement EUROCONTROL has developed safety data reporting to identify key risk areas at European level…

7. Criminal Offences Gross negligence Omissions Slips Lapses Mistakes Violations LAWS LAWS Procedures Proactive Management Procedures Proactive Management Establishing a Just Culture unintentional Management Statement in Safety Policy deliberate deliberate

8. ATM Safety System Maturity in ECAC States • Independent maturity assessment system • Applied across ECAC Region • Now being expanded to neighbouring States

9. Reactive Proactive Performance-based Approach to Managing Risk Risk Assessment New Systems Recruitment/ Selection Procedures Operational Processes Incident Reporting Safety Surveys and Follow-up Risk Assessment ATM Procedures Training Incident Investigation Interface ATS CNS, AIM, Airports Competency Checks Lessons Learnt CNS /AIM Maintenance Procedures Risk Assessment Airspace Changes Refresher/ Advanced Training Risk Assessment Software Changes Risk Assessment and Mitigation Emergency Procedures A systematic approach to the management of safety

10. Risk Assessment and Mitigation - 1 • Empirical methods of risk assessment no longer sufficient • Systems more complex – failure modes more difficult to identify • Mitigation methods are more complex too – and more costly • Performance-based approach to mitigation is needed – what are the design targets to be met?

11. Risk Assessment and Mitigation - 2 • Transparency is also required by those who will: • Own and operate the system • Ultimately rely on the safety of the system • Bear liability if the system fails • A formal, structured and visible approach is the only answer • It is required by ESARR 4 and the EC’s Common Requirements for ANS provision • A risk classification scheme is a necessary start point for the decision-making that must follow

12. Risk Management • Risk Management is primarily… • …a task for the service provider • The provider / operator manages the system and its hazards • Risk management processes are conducted as part of a Safety Management System • Legal requirement for service providers to conduct risk assessment and mitigation in relation to the implementation of changes to the ATM system This is the approach being implemented in Air Traffic Management in Europe

13. PROJECT DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION ACCEPTANCE Risk Management This is the sort of process required in ESARR 4 ... RISK ASSESSMENT AND MITIGATION ACTIVITIES

14. PROJECT PROJECT DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION RISK ASSESSMENT AND MITIGATION ACTIVITIES RISK ASSESSMENT AND MITIGATION ACTIVITIES ‘SAFETY CASE’ ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION ACCEPTANCE ACCEPTANCE Risk Management This has to be done by the provider...

15. PROJECT PROJECT DETERMINATION & SPECIFICATION DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION INSTALLATION AND TRANSITION OPERATION OPERATION RISK ASSESSMENT AND MITIGATION ACTIVITIES RISK ASSESSMENT AND MITIGATION ACTIVITIES ‘SAFETY CASE’ ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables Risk Assessment and Mitigation Deliverables REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION ACCEPTANCE ACCEPTANCE Risk Management But what about this ? WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?

16. EUROCONTROL The provider… The regulator… In some cases, the provider decides about the change… … using risk assessment and mitigation process to support its internal decision-making. • Regulators may identify new systems and changes… • … to be directly accepted by the regulatory authority through a formal acceptance (or approval) • This is possible if: • The provider’s process is demonstrated to be effective, • Enough safety oversight is focused on these processes (e.g. by means of audits) • The Regulator makes the final decision on the acceptability of the system to go into operation • The review of the ‘safety case’ provides the Regulator with evidence to support his decision Acceptance of new systems and changes

17. WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ? EUROCONTROL The ESARR 1 Approach • The ESARR 1 process for the safety oversight of changes to the ATM system: • Is implemented by the Regulator by considering results from the risk assessment and mitigation process conducted by the provider • Defines a minimum category of changes, whose safety case must be reviewed by the Regulator… …Based on the severity of the hazards identified by the provider in relation to the change • Provides the regulator with discretion to review other changes

18. WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ? EUROCONTROL Planned Change (new system or change to existing system) Implementation by the provider of the change (as accepted by the regulator) Provider conducts risk assessment and mitigation and produces a ‘safety case’ Yellow = provider Red = regulator The Role Of Oversight REGULATOR REVIEWS SAFETY CASE Acceptance by the regulator Additional Safety Conditions imposed REGULATOR APPLIES DIFFERENT APPROACH DEPENDING ON THE CHANGE Major Minor Accepted through ATM provider’s procedures (which are subject to regulator’s auditing) REGULATOR CONDUCTS SAFETY REGULATORY AUDITS • MAJOR = • Those changes whose assessment of the potential effects of hazards on the safety of aircraft, conducted by the provider in accordance with ESARR 4, identifies hazards with potential to lead to an accident or serious incident • Other changes that the Regulator considers appropriate to review

19. Summarising the Approach to Risk Management European ATM service providers are required to implement risk assessment and mitigation as part of their SMS: • Risk assessment and mitigation processes are subject to regulatory auditing as any other safety-related process • In addition, the Regulator will specifically review the results of these processes in relation to, at least, the most critical safety-related changes • The implementation of these changes will be subject to regulatory acceptance based on the results.

20. Risk Classification Scheme • We now have a severity classification scheme for the identification of the effects of ATM/CNS related hazards on the safety of aircraft. (EC law) • We also have a risk classification scheme with a maximum tolerable probability for ATM directly contributing to accidents in the ECAC region (severity class 1) ….but • maximum tolerable probability for the severity classes 2 to 5 have still to be developed. • States, EC and EUROCONTROL acting together to complete and update those probabilities, • Development of regulatory material for the establishment of a quantified risk classification scheme at regulatory level .

21. Identifying Tolerability of Change Hazard identification. Safety target likelihood effects Severity Severity of the effect No effect Catastrophic Major Average Minor 1 2 3 4 5 I II Likelihood III IV V VI Risk Mitigation Tolerable? no yes Continue the design Safety objectives

22. Conclusions - 1 • Performance-based ATM framework… We are on the way… good progress being made. Experience so far… • A performance-driven approach requires: - • Data (occurrences, maturity etc.) • “Just Culture” – overcoming inhibitors to progress • A measurement system, harmonised globally • Analysis capability • Key Performance Indicators (ultimately)

23. Conclusions - 2 • We’ve had a risk-based approach to the management of safety for decades, but…. • The risks are more difficult to identify now • Move from “historic” to “predictive” risk assessment • A formal, visible assurance methodology • We need systems to measure the risks before and after changes to the system (was mitigation successful?) • A fully functioning SMS will provide the tools to do the job

24. Conclusions - 3 • Global needs in safety: - • A common approach to safety – management and regulation • Common minimum levels of safety • Availability of information on which to base a performance-driven approach • Common safety “language” – terms, taxonomy and appreciation of risk • The correct balance between State functions and those of other stakeholders

25. Peter StastnyHead of Safety Regulation UnitEUROCONTROLpeter.stastny@eurocontrol.intICAO Montreal, Thursday 29 March 2007 Ensuring the Safety of Future Developments