Systems Engineering Approach to MPS Risk Management

1. Systems Engineering Approach to MPS Risk Management Kelly Mahoney mahoney@jlab.org Presented at the Workshop for Machine Protection in Linear Accelerators June 8, 2012

2. Systems Approach (from Tuesday’s talk) • Top-Down • Encompasses all aspects of a technical project • Focus on overall facility mission and goals • Overall context for development of systems under specific standards, e.g. IEC 61508, 61511, 62062,… • Accelerator is a system of systems • Similar lifecycle activities apply to all subsystems – rigor depends on risk under consideration. • Assumptions under one analysis become requirements to another system • Should be tracked

3. System Engineering Processes Agreement Process Organizational Process Project Process Technical Process 80/20 Rule Applied to Systems: 80% of system errors are introduced in the requirements, 20% in all remaining lifecycle stages. 80% of a project’s committed cost are determined during the 1st 20% of actual cost (Requirements + first stages of Architectural Design) Cost to correct incorrect/incomplete requirements increase by an order of magnitude for each major project activity. Stakeholder Requirements Definition Process Transition Process Requirements Analysis Process Validation Process Operation Process Architectural Design Process Maintenance Process Implementation Process Verification Process Disposal Process Ref. IEC15288/12207/ INCOSE Systems Safety Handbook CERN MPS Workshop 6-8 June, 2012

4. Safety Risk Management Systems Assurance Software Assurance Cyber Security Assurance Identify Hazards Identify Hazards Identify Hazards Assess Risk Assess Risk Assess Risk Establish Controls Establish Controls Establish Controls Implement Controls Implement Controls Implement Controls Maintain and Assess Maintain and Assess Maintain and Assess CERN MPS Workshop 6-8 June, 2012

5. Integrated System Risk Management Systems Assurance • Horizontal link of controls, assumptions, constraints • Functional testing, Software QA, defensive programming, physical security, … • Central management of hazards and risks. • Applies to all safety functions • Personnel Safety • Beam Containment • MPS • Common high level requirements and assumptions; as well as assessments. Identify Hazards Assess Risk Establish System Level Controls Establish Software Controls Establish Security Controls Implement System Level Controls Implement Software Controls Implement Security Controls Maintain and Assess CERN MPS Workshop 6-8 June, 2012

6. Integrated System Risk Management Systems Assurance • Common Requirements Among Standards: • Management Requirements • Competency in each specialty area • Graded Approach to system design, mitigations, and management based on risk • Hazard and Risk Assessment • Configuration Management Identify Hazards Assess Risk Establish System Level Controls Establish Software Controls Establish Security Controls Implement System Level Controls Implement Software Controls Implement Security Controls Maintain and Assess CERN MPS Workshop 6-8 June, 2012

7. Cyber Security Risk • Not well defined in current safety management practices • Large emphasis on control system cyber security • US NIST Common Risk Evaluation Areas • Risk to Integrity • Risk to Availability • Risk to Confidentiality • Latest version of IEC61508 attempts to address cyber security CERN MPS Workshop 6-8 June, 2012

8. Cyber Security Risk • Risk is defined in terms of ‘vulnerability’ • Consequences are same as identified in hazard analysis • Failure modes include malicious intent by internal or external party • Mitigations • Staff training and security awareness • Physical security (limited access) • Least Privileges/Authentication • Segmentation • Passive monitoring • Defensive/Fault Tolerant programming • Forensic capability • Intrusion Response Plan • Resources for control system cyber security • IEC 62443 Security for industrial process measurement and control • ISA S99.01 Security for Industrial Automation and Control Systems • US NIST “Special Publicaiton 800-53.” Recommended Security Controls for Federal Information Systems and Organization • US ICW-CERT http://www.us-cert.gov/control_systems/ics-cert/ • ENISA Protecting Industrial Control Systems Recommendations for Europe and Member States CERN MPS Workshop 6-8 June, 2012

9. JLab Controls Cyber Security • Working to establish controls cyber security program • Controls Cyber assurance program in process • Covers all controls • Risk Based Management CERN MPS Workshop 6-8 June, 2012

10. JLab Global Risk Assessment Method • Started as software risk assessment tool • Applicable to all aspects of risk management • Developed by team with representatives of all enclaves at JLab • Safety Systems (facilitator) • Network and Infrastructure (Cyber Security) • Business Computing and Information Systems • Quality Assurance • Accelerator Controls and Networking • Experimental Physics • Physics Computing and Data Management • Chief Information Officer/Chief Information Security Officer • Covers ALL software – from Experiment Data to FPGAs • Now used as basis for configuration management • Assurance process defines minimum activities for a given risk level. Does not dictate how. CERN MPS Workshop 6-8 June, 2012

11. JLab Global Risk Assessment Method • Six Areas • Direct Risk of Financial Loss • Direct Risk of Loss of Tangible Property • Direct Risk of Harm to People • Direct Risk of Harm to the Environment • Direct Risk of Loss of Mission • Direct Risk of Regulatory Body Intervention • Each subject evaluated in an FEMA type scenario • Each of the six areas are assigned a score 0-5, based on predefined unmitigated consequences. CERN MPS Workshop 6-8 June, 2012

12. JLab Global Risk Assessment Method • Score is evaluated on BOTH max value of single category AND sum of all scores • Some risks that were below the radar now pop up as more important • Because the system owner evaluates the risk, they are invested in the process • Evaluator determines risk acceptance level of unmitigated and mitigated risk. • Intolerable • Unacceptable • Tolerable • Acceptable • Amazing agreement between evaluation scores and risk acceptance levels among different enclaves. CERN MPS Workshop 6-8 June, 2012

13. Functional Risk Assessment Methods Used for JLab MPS Safety Functions • Event Tree • Risk Matrix • Risk Graph • Layer of Protection Analysis All of the above can be used to assign a SIL level to a safety function. CERN MPS Workshop 6-8 June, 2012

14. Conclusions • Systems approach allows early identification and mitigation of operational risks • Same approach can be used for all safety related systems • Correct Requirements are critical for correct and efficient implementation of a protection system. • JLab Global Risk Assessment tool can uncover risks that fall below radar in other assessments • SIL methods can be used to manage MPS safety functions’ CERN MPS Workshop 6-8 June, 2012

15. Additional Slides: CERN MPS Workshop 6-8 June, 2012

16. MIL-STD-882E System Safety Ref. MIL-STD-882E CERN MPS Workshop 6-8 June, 2012

17. 882 E Software Safety Criticality Matrix Ref. MIL-STD-882E CERN MPS Workshop 6-8 June, 2012

18. Software Assurance CERN MPS Workshop 6-8 June, 2012

19. A Note on Safety Integrity Levels (SILs) • A Safety Integrity Level applies to a mitigation function performed by a system. • Individual SILs are determined by the difference between (unmitigated risk + risk reduction of other safety layers or functions) and acceptable risk goal. • Examples: MPS Safety Requirement: Prevent catastrophic loss of two or more superconducting dipole magnets due to a beam loss event. Other Layers SF1: CERN MPS Workshop 6-8 June, 2012

20. IEC61508Lifecycle Model Overall planning Overall operation and maintenance planning Overall safety validation planning Overall installation and commissioning planning Concept Overall scope definition Analysis Phase Hazard and risk analysis Overall safety requirements Safety requirements allocation Safety-related systems: E/E/PES Safety-related systems: other technology Realization External risk reduction facilities Realization Realization (see E/E/PES safety lifecycle) Realization Phase Overall installation and commissioning Back to appropriate overall safety lifecycle phase Overall safety validation Overall operation, maintenance and repair Overall modification and retrofit Decommissioning or disposal 16 Operations Phase

21. IEC Safety Allocation USPAS June, 2004