160 likes | 315 Vues
Verification and Validation. To ensure quality in expert system verificaion and validation are important steps in the development lifecycle. Based on Chap. 16: The Engineering of Knowledge-based Systems: Theory and Practice, A. J. Gonzalez and D. D. Dankel. Errors may stem from:
E N D
Verification and Validation To ensure quality in expert system verificaion and validation are important steps in the development lifecycle. Based on Chap. 16: The Engineering of Knowledge-based Systems: Theory and Practice, A. J. Gonzalez and D. D. Dankel
Errors may stem from: • Not Complying with the system specs. • Semantic and syntactic errors • Incorrect representation of the knowledge domain
Verification and validation in expert systems is different from that of conventional expert system • ES are not purely objective • Uncertainty is accepted and may even be encouraged • ES output cannot be easily verified through experiments • Expert (or group of experts) decide on the accuracy of the system
Verification The adherence to the system specification and correct implementation process.
Compliance with specs • Knowledge representation • Reasoning technique • Modularity • Interfaces • Explanation facility • Performance • Maintainability • Security
Redundant rules Identical rules or rules that have the same meaning: If a is true and b is true then the temp is -40 0 C If a is true and b is true then the temp is -40 0 F
Conflicting rules Identical premises with conflicting conclusions: If a is true and b is true then x If a is true and b is true then ~x
Subsumed rules • All the premises of one rule is a subset of another rule, and both have identical conclusion: If a is true and b is true then x If a is true and b is true and c is true then x
Circular rules Conclusion of rule1 leads to rule2 firing, whose conclusion leads to rul1 firing: If a is true and b is true then x and y If x is true and y is true then a and b
Unnecessary IF conditions Two rules have identical conclusion, but one of the premises is conflicting: If a is true and b is true and ~c is true then x If a is true and b is true and c is true then x
Dead-end rules A rule whose action is not one of the conclusions of the system or whose conclusion is not used in the premise of any other rule.
Missing rules Dead end rules could be caused by the removal of rules or the failure of the knowledge engineer to include a rule.
Unreachable rules The premise of the rule will never ever be true.
Validation Checks to see if the system is correct, i.e. the output is consistent with input.
Validation issues • What is being validated? • Method • Criteria • When should validation occur?