Rethinking Risk Analysis

1. Rethinking Risk Analysis Tony Cox MORS Workshop April 13, 2009

2. How to better defend ourselves against terrorists?Top-level view

3. Elements of smart defense • Anticipate attacker actions, reactions • What can they afford to do? When? • What is their best response to our actions and defenses? • Allocate resources and countermeasures to protect targets and to deter attacks • Adapt to new information and intelligence • Reallocate effectively; hedge bets Risk scoring does not do these things very well How can we do better?

4. Other defenses • Secrecy and randomization • Deception, decoys, disinformation • Infiltration, counter-intelligence • Detect and interdict at attack-planning stage • Rapidly recognize, respond, contain • Preparation, excecution

5. Our focus: Attack-Defense Games • Defender allocates resources, countermeasures • Attacker decides what to do, given what the defender has done • Could iterate, several layers deep (chess) • Attacker and defender receive consequences • How can Defender minimize loss?

6. “Backward chaining” paradigm for defensive risk management • Envision:What might go wrong? • E.g., secure facility compromised or damaged • Analyze:How might it happen? How likely is it? • Identify alternative sets of sufficient conditions • Path sets, minimal path sets, dominant contributors • Recursive deepening (fault tree analysis) • Quantify relative probabilities, total probability • Assess:How bad are the consequences? • Manage risk: • Document it. • Risk = Threat x Vulnerability x Consequence (?) • Request/allocate resources to reduce risks (biggest first)

7. TVC paradigm • Risk = TVC • Threat = relative probability of attack • Reflects attacker’s intent, capability, timing decisions • Budget and resource constraints? Opportunity costs? • Vulnerability = probability that attack succeeds, if attempted • Could there be partial degrees of success, based on consequences? • Consequence = defender’s loss from successful attack • Risk management: Allocate resources to defend biggest risks first (TVC prioritylist)

8. Why isn’t TVC used in chess? • Or any other game? • Or in other risk management settings where experts (or programs) compete for prizes?

9. Improvement: Focus on changes • Risk = TVC should not drive action. • Risk = (T)(V)(C) is more useful • Risk management decisions: Allocate resources to biggest risk reductions first

10. Improvement: Focus on changes • Risk = TVC should not drive action. • Risk = (T)(V)(C) is more useful • Requires a predictive (causal) risk model: action  V T  C • How do our actions affect attacker’s? • Risk management decisions: Allocate resources to biggest risk reductions first

11. Key Challenge 1How to usefully predict T, V, C for alternative interventions?

12. Key Challenge 1How to usefully predict T, V, C for alternative interventions?Expert elicitation?Modeling?

13. Key Challenge 2How to validate that predictions and recommendations are useful?

14. Attacker’s view: “Forward chaining” paradigm • If I prepare for (or launch) attack A now… • What will I learn? (Value of information) • What opportunities must I give up? What will I gain? • What risks will I incur? (Detection/interdiction) • What direct value will it produce? (Value of damage) • How to do it? • Plan attack (including preparation steps) • Top-down: Select approach, evaluate, iteratively improve • Simulate/predict results, improve/refine/test plan • What course of action is most valuable now? • Assuming optimal future actions

15. http://www.dtic.mil/ndia/2008homest/landsberg.pdf

16. Receive consequences Make decisions http://www.dtic.mil/ndia/2008homest/landsberg.pdf

17. Receive consequences C T V Attacker’s investment and plans Defender’s investments http://www.dtic.mil/ndia/2008homest/landsberg.pdf

18. Minimax: Competing optimization Receive consequences C T V Attacker’s investment and plans Defender’s investments http://www.dtic.mil/ndia/2008homest/landsberg.pdf

19. Paradigm clash • What happens when a forward-chaining attacker meets a population of backward-chaining (or TVC) defenders? • Concern: Attacker wins too much! • Do defenders have a better way to outsmart attackers? • “Outsmart” = anticipate and prepare for what the attacker will do next • Yes: Minimax, not risk-scoring

20. Some challenges and limitations of risk scoring

21. Technical challenges • Uncertainty about (T, V, C) for each target • Correlated uncertainties (across targets) • Attacker behaviors, selection of targets • Countermeasure effectiveness • Consequences of successful attacks • How to optimize sets (portfolios) of defenses? • Taking dependencies into account • How to optimize resource allocation across opportunities • For defender and attacker

22. How to treat uncertain (T, V, C)? • RAMCAP™: Treat T, V, C as random variables, use their expected values • BTRA: Use expert elicitation, Monte-Carlo simulation uncertainty analysis • Minimax optimization: Model the uncertainties about (T, V, C) in more detail • What must be resolved to determine (T, V, C)?

23. Facility A: Risk = ?

24. Facility A: Risk = ? “no snow” “snow”

25. Facility A: E(T)E(V)E(C) = 0.24

26. Facility A: E(T)E(V)E(C) = 0.24

27. Facility B: Risk = ?

28. Facility B: E(T)E(V)E(C) = 0.16

29. Expected T, V, C values are irrelevant for predicting risk

30. Expected T, V, C values are irrelevant for predicting risk

31. Expected T, V, C values are irrelevant for ranking risks

32. Expected T, V, C values are irrelevant for ranking risks

33. E(T)E(V)E(C)  E(TVC)

34. Lesson 1: Don’t use expected values

35. Another example: E(T)E(V)E(C) may over- or under-estimate true risk, E(TVC) • E(T) = E(V) = E(C) = 0.5. What is E(T)E(V)E(C)? • Assume Pr(V = 1) = Pr(V = 0) = 0.5, so E(V) = 0.5 • Then E(T)E(V)E(C) = 0.125 • But, if T = C = V, then E(TVC) = 0.5 • If T = C = (1 - V), then TVC = 0. • Dependencies and correlations matter!

36. Lesson 2: No other summary measure works, either! (Need joint, not marginals)

37. Challenges • Threat depends on what the attacker knows about vulnerability and consequence… • and on how he uses that knowledge to select (and plan, and improve) attacks. • Threat depends on vulnerability and consequence • Positive correlation  multiplication is wrong

38. How to treat uncertain (T, V, C)? • RAMCAP™: Treat T, V, C as random variables, use their expected values • Use Monte-Carlo uncertainty analysis

39. C T V Event tree, MC simulation http://www.dtic.mil/ndia/2008homest/landsberg.pdf

40. Challenges • Threat depends on: • What the attacker knows about vulnerability and consequence… • How he uses that knowledge to choose, plan, and improve attacks. • Requires modeling planning/optimization • Threat depends on vulnerability and consequence • Positive correlation  multiplication is wrong • Dependency is based on decision-making • Modeled by decision trees, not just event trees

41. How to meet these challenges? • Model the uncertainties about T, V, C • Simulate attacker decisions under uncertainty • Outsmart the attacker

42. How to treat uncertain (T, V, C)? • RAMCAP™: Treat T, V, C as random variables, use their expected values • Use Monte-Carlo uncertainty analysis • Alternative: Model uncertainty in more detail • Why is T uncertain? • Because V and C affect T in uncertain ways • Because V and C are uncertain • Develop decision tree or influence diagram • Model: Pr(T = 1 | V, C) = E(T | V, C) • Countermeasures  (V, C, info.)  T

43. Risk from an uninformed (“blind”) attacker = 0.4 “no snow” “snow”

44. Risk from a better-informed attacker, who needs E(V)C > 0.8 to attack: 0 “no snow” “snow”

45. Risk from an informed attacker, who needs VC > 0.8 to attack, is: 0.4 “no snow” “snow”

46. Risk from an adaptive attacker, who waits to attack until V = 1, is: risk = 1! “no snow” “snow”

47. Lessons • Risk (and threat) can be 0, 0.4, or 1, depending on what the attacker knows (or believes) about V and C • Not on what we know about the attacker, or about V and C • “Threat assessments” based on our knowledge can be misleading • A valid “threat assessment” requires considering the attacker’s whole decision. • Attacker’s own assessment: T = 0 or T = 1

48. Example: Misguided threat assessment • Assume that we know that: • Attacker attacks (T = 1) if and only if he knows that success probability = 1 (V = 1). (Else, T = 0.) • Common knowledge: True success probability = 0.5 • Does attack probability = 0? • Not necessarily! (Misleading inference) • Suppose attacker attacks if and only if he first gets inside help that makes V = 1. (Else, V = 0) • Pr(succeeds in getting inside help) = 0.5 = E(V) = V • Then T = Pr(attack) = 0.5, not 0 • Threat and vulnerability assessors needs trees, not numbers, to communicate essentials about adaptive attackers and future contingencies.

49. Threat = plan tree, not number • Threat depends on what attacker knows or believes about vulnerability and consequence… • and on how he will use that knowledge to improve attacks (“attack plan”) • What will he do next? • What is his whole decision tree? • How do threat and vulnerability co-evolve? • No number can tell us all this!

50. Which threat is greater, A or B? attack hazard rate B A time