Traditional and Current Risk Analysis Tony Cox MORS Workshop April 14, 2009
Traditional Risk Analysis • Risk assessment: How bad is it? • Hazard identification: What could go wrong? How? • How likely is it? • Fault trees, event trees, PRA, Monte Carlo simulation • So what? • Consequence modeling and evaluation • Risk communication: What to say about it? • Document and compare risks • Risk = Threat x Vulnerability x Consequence (?) • Red, yellow, green? • Risk management: What to do about it? • Request/allocate resources to biggest risks first? • (Risk attribution: Who to blame for it, how much?)
Two ways to manage risks • Choose actions to optimize risk reduction (subject to constraints) • Budget constraints • Interactions among threats, vulnerabilities, consequences, countermeasures • Minimax: Anticipate attacker’s response • Identify, document and rank concerns, then tackle biggest ones first.
Two ways to manage risks • Choose actions to optimize risk reduction (subject to constraints) • Budget constraints • Interactions among threats, vulnerabilities, consequences, countermeasures • Minimax; Anticipate attacker’s response • Identify, document and rank concerns, then tackle biggest ones first. • This talk: First way is better. Second can be surprisingly bad.
Terrorism risk assessment • Risk matrices • Red, yellow, green, high, medium low • Risk scoring formulas • Risk ranking and priority lists • Risk simulation models • Risk optimization models • Attacker-defender models, game theory • Optimize resource allocations
MIL-STD-882c, January, 1993 http://www.weibull.com/mil_std/mil_std_882c.pdf
Source: FAA, 2007 www.faa.gov/airports_airtraffic/airports/resources/advisory_circulars/media/150-5200-37/150_5200_37.doc
Now, everyone’s doing it • National and international standards • Guidance documents • Computer and IT security • Threat, vulnerability, consequence ratings for terrorism threats • Compliance programs • Training • Certification programs
Example risk matrices Swedish Rescue Service U.S. FHA Supply Chain Digest Australian Government
Should B outrank A? • A • B
Not necessarily! • A Isorisk contour • B
How bad can misrankings be? • Misrankings always exist • for any coloring and smooth, downward-sloping indifference-curve contours • Unavoidable • Up to 100% of points can be misranked (!) • if frequency and severity are negatively correlated • More than three colors: Spurious resolution • inconsistent with any quantitative model, increases misrankings • Common in practice
Some other problems • Ambiguous (and mysterious) rating scales • “Frequency” is not well-defined • “Severity” is not well-defined • “Risk” (and risk reduction) are not well-defined • Recommended decisions are often bad • Budget constraints? • Diversification? • Optimization?
Mysterious definitions “Almost certain: Is expected to occur in most circumstances Likely: Will probably occur in most circumstances Possible: Might occur at some time Unlikely: Could occur at some time Rare: May occur only in exceptional circumstances” www.health.gov.au/internet/main/publishing.nsf/Content/mental-pubs-n-safety-toc~mental-pubs-n-safety-5~mental-pubs-n-safety-5-7
Some other problems • Ambiguous (and mysterious) rating scales • “Frequency” is not well-defined • Is MTBF ~ U[0, 8] more frequent than MTBF = 4? • No way to define “frequency” so smaller is better. (MTBF= 4 is preferred if mission life is 3 years, but not if it is 5 years) • “Severity” is not well-defined • “Risk” (and risk reduction) are not well-defined • Recommended decisions are bad • Budget constraints? • Diversification? • Optimization?
Ambiguous rating scales and definitions Descriptions are not mutually exclusive Suppose that controls are in place to prevent an attack, but they are ineffective during snow storms
Suppose Pr(H) = Pr(L) = 1/2? • Ratings do not handle uncertainty • Example: How to rate “likelihood” of an event judged equally likely to be “H” or “L” • (Risk matrix-based standards never address this question.)
Limitations of “Likelihood” ratings • Example: Suppose likelihood ratings are: • Low (L): 0 p 0.4 • Medium (M): 0.4 < p < 0.6 • High (H): 0.6 p 1 • Then the “likelihood” of an event that is equally likely to be “H” or “L” should be…. • “L” if the two equally likely values for p are (0, 0.7) • “M: if the two equally likely values for p are (0.3, 0.7) • “H” if the two equally likely values for p are (0.3, 1) • Need numbers to know what to do.
100% probability of loss 28 40% probability of loss 40 Why should an expected loss of 0.40*40 = 16 outrank a sure loss of 28?
100% probability of loss 28 40% probability of loss 40 No way to objectively classify highly uncertain impacts.
From risk rating to risk management: Setting risk management priorities without costsor budgets Costs? Budget?
Why is 100% probability of minor impact rated the same as 100% probability of critical impact? (Range compression)
How should we rate a risk with 5% probability of moderate impact and 95% probability of minor impact?
Summary so far… • Risk matrices have some problems • “Frequency” is usually undefined • “Severity” is usually ambiguous • Risk ratings can be worse than useless • Misranking can be worse-than-random • Recommendations are often nonsensical • Give higher management priority to smaller risks
Can risk formulas do better? • Remove artificial discretization • Allow smooth indifference curves … but the fundamental problems remain
Risk formulas Examples: • Risk = Threat Vulnerability Consequence • All values are expected values (RAMCAP) • Risk = frequency severity • Risk = jwjxj, • xj = level of bad attribute j • wj = importance weight for attribute j • Risk (expected loss) = f(attributes)
Example: Additive scoring rule Bioterrorism risk scoring (Macintyre, 2006): • Probability of attack = ease of procurement + ease of weaponization + history of use • Impact = lack of preventability of disease + lack of treatability • Score each factor as: 0 = no, 1 = low, 2 = high • Priorityscore = Probability + Impact
Example: Additive scoring rule Implications of: • Priorityscore = Probability score + Impact score • (0 + 2 = 2 + 0) • (unobtainable agent, untreatable effect) ~ (obtainable agent, treatable effect) • Zero risk ~ positive risk bad advice!
Plotting - Establish probability scale on y-axis - Establish impact scale on x-axis - Priority regions are set by the risk assessors Red - Highest PriorityYellow - Medium PriorityGreen - Low Priority http://www.mitre.org/work/sepo/toolkits/risk/procedures/RiskPlotting.htm l
TVC paradigm • Risk = threat x vulnerability x consequence • Threat = relative probability of attack • Reflects attacker’s intent, capability, decisions • Budget and resource constraints? Opportunity costs? • Vulnerability = probability that attack succeeds, if attempted • Partial degrees of success? • Consequence = defender’s loss from successful attack • Sum over multiple risks to get total risk • Risk management: Allocate resources to defend biggest risks first • TVC prioritylist
Summing risks • “The risk associated with one asset can be added to others to obtain the aggregate risk for an entire facility… [and] can be aggregated and/or compared across whole industries and economic sectors. This is precisely the goal of DHS.” (RAMCAP Framework) • Is this a good idea? • No: Risks should not be added! • T1V1C1 + T2V2C2 is not valid
Risks are not additive • Let success prob. for “attack via front door” = V1 = 0.5; and let V2 = 0.5 for “attack via back door”. (Let T = C = 1 for both.) • If these two vulnerabilities, 0.5 and 0.5, are independent, then what is the total risk? • It is not T1V1C1 + T2V2C2 = 0.5 + 0.5 =1. • It is 1 – 0.5*0.5 = 0.75 • If the two vulnerabilities are dependent, then total risk can be anywhere between 0.5 (if r2 = 1)and 1 (if r2 = -1)
Doubts about additivity Two attackers, each can grab either the $10 nearest it, or the $8 in the middle. attacker A: $10 $8 $10 attacker B • Which asset should we protect, if we can only afford to protect one?
Doubts about additivityattacker A $10 $8 $10 attacker B “Relative attractiveness” solution: • For each attacker, Pr(grab $8) = relative value of asset = 8/18 = 4/9. • Pr(grab $10) = 10/18 = 5/9 • Sum of threats: • 4/9 + 4/9 = 8/9 for $8 > 5/9 for each $10. • So, defend the $8!
Fundamental limitations of risk scoring and ranking Risk formulas and ranks do not exploit • Correlations • Diversification • Interactions, portfolios of countermeasures • Scale of risk management decisions • Distributed risk management • Budget, resource constraints
Risk scores fail to diversify • Any risk formula that ranks or scores identical prospects identically fails to optimally diversify portfolios of risk-reducing measures
Risk scores fail to diversify • Example: We can afford to protect 10 sites. • Benefits depend on unknown state of world • Two equally likely states (attacks on A vs. B sites): • Protecting a type A site yields (0 or 100) benefits • Protecting a type B sites yields (100 or 0) benefits • Optimal risk-averse decision: Protect 5 of each type of site • Benefit = 500 (variance = 0) • Risk-scoring: Protect 10 of higher-scoring type • Benefit = 0 or 1000 (maximum-variance)
Risk scores ignore interactions • Risk formulas that score each countermeasure based on how much it reduces risk do not create optimal portfolios of risk-reducing measures. • Budget constraints imply that best portfolio cannot necessarily be represented as a priority list.