Abstract

1. First “Watertime” Steering Committee meeting and stakeholder workshop Decision Models – An Analytical Framework for the Watertime ProjectD. GrahamSchool of Computing and Mathematical SciencesUniversity of GreenwichPark RowLondon SE10 9LSUKE-mail: D.Graham@gre.ac.ukhttp://www.cms1.gre.ac.uk

2. Abstract This presentation suggests a possible framework for the encapsulation of the decision making process for the Watertime project. The final outcome maybe a computerised model, but the process advocated is not so prescriptive, and involves the production of a “paper model” as mediating representation between the knowledge acquired and any computerised system. This paper model may suffice in terms of the project’s goals. Keywords Decision making models, Decision trees, knowledge acquisition, Systemic Grammar Networks.

3. 1. Introduction Whatever the form of the final outcome (computer system or paper model), the results will essentially be the representation of the decision process for Watertime and associated knowledge.

4. 2. Proposed framework The first activity required is knowledge acquisition. Different authors present methodologies with varying stages of knowledge acquisition, but fundamentally they all involve; the identification and conceptualisation of requirements and problem characteristics, formalising these into some mediating representation scheme, implementation, and final testing and validation. (Graham & Barrett, 1997). Knowledge acquisition can be machine-aided or human-labour oriented. Johnson and Johnson’s methodology (1987a), enhanced by Graham (1990), propose a three phase knowledge elicitation process based around semi-structured interviews.

5. The first phase is to perform a broad, but shallow survey of the domain. This allows the elicitor to become oriented onto the domain, so a more flexible approach can be taken. This type of horizon broadening is a standard approach in social science research. Once this shallow trawl of the domain has been done, the second phase requires that a more detailed task analysis is performed by the elicitor, focusing on areas of interest. The structure of the interview uses a teachback technique to traverse the domain and validate elicitor understanding with the result that the elicitor progressively refines the model of the expert’s competence.

6. This model is qualitatively drawn up and uses a mediating representation, Systemic Grammar Networks. These are a context free, qualitative representation, which can be used as a tool for systems design, but their use does not imply the final application of any particular knowledge engineering shell or methodology. SGNs have been used in many domains including oncology, pcb (printed circuit board) design, and fault diagnosis.

7. - ST506MFM - Wait FS5000; ____ - FS5000 = not free FS2304_port0 - FS2304_port0 = not free _ Wait ____ - Wait ____ - EDSI FS2304_port1 - FS2304_port1 = not free - Wait ____ - ST506RLL FS2304_port2 - FS2304_port2 = not free - Wait ____ - SCSI FS2304_port3 - FS2304_port3 = not free FT _____ _ Repair ___ - DUT_scan = FAIL _ - ST506MFM ______________________ - FS5000 = free Result - ST506MFM _ Pass to ___ - DUT_scan = PASS _ - FS5000 = not free engineer - FT_rig_message=FAIL - FS2304_port = free _______________________ _ - EDSI - FS2304_port1 = free _______________________ _ Store ____ - DUT_scan = PASS _ - ST506RLL - FT_rig_message=PASS - FS2304_port 2 = free _ - SCSI - FS2304_port3 = free Figure 1: SGN (version 4) based on final test results (Graham, 1990: 146).

8. Above (figure 1) is an example of an SGN produced for computer hardware fault diagnosis and repair, specifically, for the final test stage of diagnosis and repair of Maxtor disc drives. A short summary of the notation used follows (figure 2). For further information on SGNs, see Bliss et al., (1983).

9. GENERAL IDEA TECHNICAL TERM NOTATION Category name; thing picked out term e.g. Wait Finest category or distinction made terminal e.g. ST506MFM Choice; difference in context of system BAR __ alternatives __ Parallel aspects; simultaneous co-selection BRA __ choices __ __ Circumstances alter cases; entry condition CON __ restriction; constraints __

10. GENERAL IDEA TECHNICAL TERM NOTATION Repeated possibilities recursion REC Greater fineness of distinction delicacy tree structures One from many possible patterns paradigm path in a network Saying what is in an item of data code e.g. FS500 value free Handy expression for a category realisation rule e.g. copy terminals Example in data of a category instantiation How categories deal with data representation Size, scale, unit of things rank described Finding a reasonable way to talk descriptive language about data Figure 2: SGNs - Summary of Ideas, Terms and Notation

11. The third phase of this approach is to validate the models drawn up from the expert with the wider expert community. The theoretical predictions of the model presented to the initial community used in the first phase, and then to a further independent population, to check the appropriateness and validity of the model which has been created.

12. 3. Application of proposed framework The broad and shallow survey may be achieved by the use of questionnaires, plus importantly, a letter of introduction to the project, to all the parties involved in the decision making. The second phase, the task analysis, is more focused and should concentrate perhaps on one of the more typical participants, based on the revelations of the first phase. Whilst the use of semi-structured interviews is the main tool recommended for the task analysis, it should include other techniques (such as observation) to gather corroborative data. The product of this second phase, Systemic Grammar Networks (SGNs), may suffice Watertime’s “paper model” requirements. This model being revised in accordance with the third and final phase, namely, validation.

13. The SGNs could however, be reproduced as decision tables and decision trees (Luger, 2001). Information theory (Shannon, 1948) could then be applied to re-structure the decision trees in relation to their information content value, leading to simpler decision trees, but still retaining the decision making process in the form of a paper model. Luger (2001) provides an example of a decision table (figure 3), and decision trees (figures 4 and 5) for Credit Risk Assessment

14. N0. RISK CREDIT DEBT COLLATERAL INCOME HISTORY ($) 1. high bad high none 0 to 15k 2. high unknown high none 15 to 35k 3. moderate unknown low none 15 to 35k 4. high unknown low none 0 to 15k 5. low unknown low none over 35k 6. low unknown low adequate over 35k 7. high bad low none 0 to 15k 8. moderate bad low adequate over 35k 9. low good low none over 35k 10. low good high adequate over 35k 11. high good high none 0 to 15k 12. moderate good high none 15 to 35k 13. low good high none over 35k 14. high bad high none 15 to 35k Figure 3: Data from credit history of loans (Luger, 2001: 373)

15. Credit history? Unknown Bad Good Debit? Collateral? Debt? High Low None Adequate High Low High riskCollateral? High risk Moderate riskCollateral? Low risk None Adequate None Adequate Income?Low riskIncome?Low risk 0 to 15k 15 to 35k Over 35k 0 to 15k 15 to 35k Over 35k High risk Moderate risk Low risk High risk Moderate risk Low risk Figure 4: A decision tree for risk assessment (Luger, 2001: 374)

16. In figure 5, an improved version of the initial tree produced from the decision table is refined using information theory, to restructure the nodes of the tree in accordance with the information content value of each property (risk, credit history, debt, collateral, income).

17. Income 0 to 15k 15 to 35k over 35k High riskCredit history? Credit history? Unknown Bad Good Unknown Bad Good Debt? High risk Moderate risk Low risk Moderate risk Low risk High Low High risk Moderate risk Figure 5: A simplified decision tree for credit risk assessment (Luger, 2001: 374)

18. 4. Conclusions In this presentation paper we have proposed a well established methodology for knowledge elicitation, plus several schemes for the representation of acquired knowledge as a paper model; SGNs, Decision Tables and Decision Trees, for the decision making process for the Watertime project. Either SGNs or decision trees are amenable to computerisation, using a Decision Tree Induction Algorithm (Quinlan, 1996) for example, which focuses solely on the decision process. Alternatively, as or a more comprehensive Knowledge-based system, attaching procedures, queries and other features inbuilt in the software, as in an expert system shell for instance. If the computerisation of the decision making process is to be consider, then it is suggested that further funding for this element of the project is investigated.

19. References [1] Bliss, J., Monk, M. & Ogborn, J (1983). Qualitative Data Analysis for Educational Research. Croon Helm. [2] Graham, D. (1990). Knowledge Acquisition: A Case Study in Computer Fault Diagnosis and Repair. PhD thesis, Brunel University. [3] Graham, D. & Barrett, A. (1997). Knowledge-Based Image Processing Systems. Springer-Verlag. [4] Johnson L. & Johnson, N. E. (1987a). Knowledge elicitation involving teachback interviewing. In: Knowledge acquisition for expert systems: a practical handbook. Kidd, A. L. (Ed.), New York, NY Plennum, pp 91-108. [5] Luger, G. F. (2001). Artificial Intelligence: Structures and Strategies for Complex Problem Solving. Addison-Wesley. [6] Quinlan, J. R. (1996). Bagging, Boosting and CN4.5. Proceedings AAAI 96. Menlo Park: AAAI Press. In: [5]. [7] Shannon, C. (1948). A mathematical theory of communication. Bell Systems Technical Journal. In: [5].