ATSSD SRG CAA (UK) Experience with Goal Based Regulations

1. ATSSD SRG CAA (UK)Experience withGoal Based Regulations Andrew Eaton National Requirements & Strategy Specialist

2. Overview • What is the ATSSD SRG CAA (UK) • Our starting point • Why we decided to change to Goal based Regulations • Our understanding of Goal Based Regulations • Changes for the Regulator • Changes for the Regulatee • Lessons learnt • Outstanding Issues

3. What is the ATSSD SRG CAA (UK) • Civil Aviation Authority (United Kingdom) • Safety Regulation Group • Air Traffic Services Standards Department

4. Our starting point • Prescriptive regulations • ICAO standard based • Modified to include EU legislation • Mostly focused on interoperability • Enhanced by UK experiences • Inspection / Tick sheet driven

5. Why we decided to change to Goal based Regulations Prescriptive regulations proved to be: • Driven by Robens & Pipa Alpha reports • Incomplete / Inconsistent • Slow to respond to new technology • Obstructive to innovation • Placing the responsibility for safety with the regulator

6. Our understanding of Goal Based Regulations • Are regulations that: • set a state to be achieved without mandating a solution • are traceable via a valid argument to a safety axiom • Or are traceable via a valid argument to a legal safety requirement • Can be very detailed/prescriptive if traceability exists.

7. SW01 assurance goal “For arguments and assurance evidence to be available which show that the risks associated with deploying any software used in a safety related system are tolerably safe.” (CAP 670, SW01 Part 2 Section 3.)

8. Assurance goal decomposition This requires assurances to be made in the following areas: • Requirements Validity • Requirements Satisfaction • Non-Interference • Requirements Traceability • Configuration Consistency

9. 1. Requirements Validity • To ensure that arguments and evidence are available which show that the Software Safety requirements correctly state what is necessary and sufficient to achieve tolerable safety, in the system context . (CAP 670, SW01 Pt 2 requirements.)

10. 2. Requirements Satisfaction • To ensure that arguments and evidence are available, which shows that the software satisfies its safety requirements. (CAP 670, SW01 Pt 2 requirements.)

11. 3. Requirements Traceability • To ensure that arguments and evidence are available which shows that all Safety Requirements can be traced to the same level of design at which their satisfaction is demonstrated. (CAP 670, SW01 Pt 2 requirements.)

12. 4. Freedom from interference by non safety functions • To ensure that functions implemented as a result of Software Safety Requirements are not interfered with by other functions implemented in the software. (CAP 670, SW01 Pt 2 requirements.)

13. 5. Configuration Consistency • To ensure that the arguments and evidence, for the safety of the software in the system context, are from: a known executable version of the software and a known set of software products, data and descriptions that have been used in the production of that version. (CAP 670, SW01 Pt 2 requirements.)

14. Underlying model of SW01

15. What SW01 is • Guidance on setting credible success criteria for judging the achievement of the regulatory objectives. • Process independent. • Lifecycle independent. • Risk focused.

16. What SW01 isn’t • It is not a software assurance standard. • It does not define a software development process. • It does not define a software assurance process to follow. • It does not reason how your assurance evidence satisfies a claim.

17. Challenges in drafting Goal based regulations • Level of detail (tend to be verbose) • Level of abstraction of the Goal • Prolific low level goals are difficult to manage • Low level goals imply regulatory risk if wrong • High level goals risk miscomprehension • Defining success criteria not solutions • Providing guidance which can not be taken to mandatory

18. Goal Based Safety Cases • Claim trees for each Goal with Arguments & Evidence • Argument design is key • Top down (argument driven), Middle out (standard driven), Bottom up (evidence driven) • Bespoke (argument drives design), COTS (argument justifies design) • Different arguments for different system components • GSN strongly preferred • Evidence filtering – pertinence to argument

19. Changes for the Regulator • Converting Inspectors to Auditors • Evaluation of arguments • Checking validity of evidence • Managing the diversity of solutions • Demands on technical knowledge • Diversity of safety arguments

20. Changes for the Regulatee • Inability to let contracts against a standard • Argument construction • Pertinence of evidence • Availability of resources / skills / knowledge • Additional workload in evaluation design options – the downside of freedom of solution

21. Outstanding Issues • Conformance/Type Assessment • Resources / Skills / Knowledge required during transition • Level of argumentation • The drive to make guidance mandatory • The process by which Goals become Prescriptive • The optimum level of Goal

22. Lessons learnt • Ensure that your understanding of a “Goal Based Regulation” is well defined • Transition needs to be planned & managed • Implement the transition top down • Be prepared to give guidance / assistance • Regulatory independence can be compromised during the transition • Expect resistance to change

23. Questions ?

24. Andrew Eaton • ATSSD,2W,Aviation House,Gatwick Airport South,West Sussex,RH6 0YR. • 01293 573504 • andrew.eaton@srg.caa.co.uk • www.caa.co.uk