The Value of Information

1. The Value of Information John Medhurst, Larrainzar Consulting Solutions Ltd Maj Ian Stanton, Dstl Ian Mitchell, Dstl

2. Value of Information • Seeks to answer the question – “What is the value of information in NBC defence” • Work for this year • Seek to understand one element of NBC decision making – the response to biological detector alarms • Use of an experiment to find out how military personnel use information to decide whether a BW attack has occurred

3. Experimental Method • Simple scenario and abstract, high-level map provided, with sensor locations and rough location of own units • Other information presented on a series of cards • Key advantages of cards • Allows us to present info in ‘bite sized’ chunks • Allows us to ensure that information is received and understood • Players required to decide when to declare ‘Precautionary Alert’ and when to declare ‘Probable Attack’. Corresponds to issuing of NBC 3 and decision to recommend initiation of medical countermeasures. • Eight Serials with Latin Square design (8 x 8), order of serials assigned randomly from Latin Square • Total of approx 550 data points if all sequences of cards accounted for – all serials carried through to end, even if decision made

5. What the Cards Look Like

6. Quantitative Results • Analysed using multivariate Probit analysis • Simple MV Probit Model • Consists of • An intercept for precautionary alert • An intercept for probable attack • A factor for each of the six card types • Gives a value which can be looked up on the normal distribution to give a probability • Intercepts can be interpreted in terms of Bayesian ‘prior belief’

9. Discussion • Although the simple model ignores elements such as pair-pair variability, these elements are what the probabilistic element of a probit model is intended to represent • The simple model accounts for 78% of the variation in P(precautionary alert) and 63% of that in P(probable attack) • Key question is how good the model is at predicting behaviour • To look at the accuracy of the simple model, we have applied the simple formula on a serial by serial basis • A sample of the results (Serials 1-4 for Precautionary Alert) are shown on the next slide

11. Effectiveness of the Simple Model • To assist in using the game as a training aid a ‘DS Solution’ has been developed for the serials • Accuracy of the simple model in predicting the ‘DS Solution’ is 75% if p>0.5 is used as the trigger • Accuracy of median human pair is slightly less • There is potential for a similar algorithm to be incorporated in the NBC BISA as a ‘wake-up’ system • Could be calibrated by use of the game as a training aid – would allow a continually updated knowledge base to support decision-making in the field

12. Conclusions • The initial aim of this phase of the study has been achieved – we have established the relative value of information in responding to a possible BW attack • There is potential for use of the experimental method as a training game, for which there is a demonstrable need • We have a simple model of human response to information, and an experimental method for estimating the effect of individual pieces of information on a decision • We also have a simple model of the response to biological alarms that has potential to be used as a decision support tool in the NBC BISA