Advisor ： Dr.Hsu Graduate ： Keng-Wei Chang Author ： Andrew K. C. Wong Yang Wang

1. 國立雲林科技大學National Yunlin University of Science and Technology • Pattern Discovery: A Data Driven • Approach to Decision Support • Advisor：Dr.Hsu • Graduate： Keng-Wei Chang • Author： Andrew K. C. Wong • Yang Wang IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS-PART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 1, FEBRUARY 2003

2. Outline • N.Y.U.S.T. • I.M. • Motivation • Objective • Introduction • Brief Description of Pattern Discovery • Data, Events, and Patterns • Pattern Discovery • Inferencing with Patterns for Decision Support • Summary and Discussion

3. Decision support nowadays is more and more targeted to large scale complicated systems and domains. Motivation • N.Y.U.S.T. • I.M.

4. Having capability of processing large amounts of data and efficiently extracting useful knowledge from the data Develop a fundamental framework toward intelligent decision support by analyzing a large amount of mixed-mode data Objective • N.Y.U.S.T. • I.M.

5. If machine intelligence could be used comfortably by the decision makers Discover multiple patterns from a database without relying on prior knowledge Cope with multiple and flexible decision and objectives Provide explicit patterns and rules associated for interpretation Render a high-speed interactive mode for information and knowledge extraction 1. Introduction • N.Y.U.S.T. • I.M.

6. Three related issues are also of concern to the decision-makers Flexibility and versatility of the pattern discovery process； Transparency of the supporting evidences； Processing speed. 1. Introduction • N.Y.U.S.T. • I.M.

7. In the seventies Quantitative basis of information measures and statistical patterns This formed the early basis of our pattern discovery approach Pattern recognition methods for discrete valued and continuous data 2. Brief Description of Pattern Discovery • N.Y.U.S.T. • I.M.

8. In the late seventies and early eighties Dimension was too large to pattern discovery Database partitioning was proposed In nineties Shift pattern recognition paradigm from the variable level to event level based 2. Brief Description of Pattern Discovery • N.Y.U.S.T. • I.M.

9. each values from its domain Generalized Event Pattern 3. Data, Events, and Patterns • N.Y.U.S.T. • I.M.

10. A Borel σ-field is the collection of all rectangles in Two advantages geometric perspective Probability measure 3.1 Generalized Event • N.Y.U.S.T. • I.M.

11. Definition 1：Consider the sample space A hypercell H of is called a hypercell if it has the form 3.1 Generalized Event • N.Y.U.S.T. • I.M.

12. Definition 2: An event in is a hyper set. Definition 3: The volume of an event is the hypervolume of the N-dimensional subspace occupied by the event. Ex:a data set D = {ω} from a sample space 3.1 Generalized Event • N.Y.U.S.T. • I.M.

13. Definition 4: The observed frequency, , of an event E in the sample space Ω is the number of data points that fall within the volume of E. Ex: as the finite set inside the volume of E then = |{X}| Definition 5: The probability of an event E is intuitively estimated by the proportion of data points contained in the event 3.1 Generalized Event • N.Y.U.S.T. • I.M.

14. Definition 6: Let Ω be the sample space and g(.) be a test statistic corresponding to a specified discovery criterion c. A pattern is an event E that satisfies the condition 3.2 Pattern • N.Y.U.S.T. • I.M. be the critical value of the statistical test at a significant level of α

15. Definition 7: An event association is a significant joint occurrence of low-dimensional events. N-dimensional event (N > 1) can be considered an event association, composed of N one-dimensional events. 3.2 Pattern • N.Y.U.S.T. • I.M.

16. Definition 8: Suppose we have a data set with sample space Ω. Pattern Discovery is the search for significant events (hypercells) in a compact subspace of the sample space Ω demarcated by the available data set D. Pattern Discovery as Residual Analysis Pattern Discovery as Optimization 4. Pattern Discovery • N.Y.U.S.T. • I.M.

17. Definition 9: The residual of an event E against a pre-assumed model is defined as the difference between the actual occurrence of the event and its expected occurrence. 4.1 Pattern Discovery as Residual Analysis • N.Y.U.S.T. • I.M. is the expected occurrence

18. Definition 10: The standardized residual of event E is defined as the ratio of its residual and the square root of its expectation Definition 11: The adjusted residual of event E is defined as 4.1 Pattern Discovery as Residual Analysis • N.Y.U.S.T. • I.M. is the variance of

19. Two pre-assumed model uniform distribution ; (concentration-driven discovery) where V is the volume of S, and M is the total number of observations. mutual independence. (dependency-driven discovery) 4.1 Pattern Discovery as Residual Analysis • N.Y.U.S.T. • I.M. is the number of data points falls into

20. C represents one of the corners of E, and L represents the lengths of the edges. Further define 4.2 Pattern Discovery as Optimization • N.Y.U.S.T. • I.M.

21. The pattern discovery problem is to The objective function O(E) is the adjusted residual 4.2 Pattern Discovery as Optimization • N.Y.U.S.T. • I.M.

22. Building Classifiers Multivariate Probabilistic Density Estimation Interpretation of Patterns Discovered Patterns as Queries for Class Data Retrieval 5. Inferencing with Patterns for Decision Support • N.Y.U.S.T. • I.M.

23. Based on the mutual information in information theory (dependency-driven discovery) information gain weight of evidence 5.1 Building Classifiers • N.Y.U.S.T. • I.M. I(.) is the mutual information result: + ; - ; 0

24. But need to estimate the conditional probabilities or know the distribution decompose if significant event associations related to and x are known 5.1 Building Classifiers • N.Y.U.S.T. • I.M.

25. Only the significant event associations discovered from the data set are used in the inference process. Thus , maximize the Conditions Using the pattern as a model, any missing values of a discrete variables can be classified 5.1 Building Classifiers • N.Y.U.S.T. • I.M.

26. The estimation of the probability density function (pdf) is a central problem in multivariate data analysis. (concentration-driven discovery) discrete pdf Estimation continuous pdf Estimation 5.2 Multivariate Probabilistic Density Estimation • N.Y.U.S.T. • I.M.

27. Definition 12: The indicator function for a event, Ei, is defined as The probability density estimate The normalization condition The discrete probability density function discrete pdf Estimation • N.Y.U.S.T. • I.M.

28. The basic idea is to estimate a kernel for each event. Gaussian kernel, its continuous and satisfies where continuous pdf Estimation • N.Y.U.S.T. • I.M.

29. To fit a kernel ψ(x) to the event E compute the mean and covariance matrix The combined pdf is estimated by where continuous pdf Estimation • N.Y.U.S.T. • I.M.

30. An exmple of continuous density estimation 5.2 Multivariate Probabilistic Density Estimation • N.Y.U.S.T. • I.M.

31. Since events was discovered, rule cane be easily extracted. association rule, form X => Y support and confidence is measured P(X,Y) and P(Y|X) 5.3 Interpretation of Patterns • N.Y.U.S.T. • I.M.

32. One pattern The Query 5.4 Discovered Patterns as Queries for Class Data Retrieval • N.Y.U.S.T. • I.M.

33. Develop a framework of data driven decision support based on pattern discovry the motivation, historical background and the rationale of our approach； a novel unified framework to define and represent mixed-mode data, the most general and common data encountered in today’s database； the theoretical basis of pattern discovery based on statistical residual and optimization principles； 6. Summary and Discussion • N.Y.U.S.T. • I.M.

34. Develop a framework of data driven decision support based on pattern discovry a novel and unified framework to represent probability models for mixed-mode data in the form of pdf； an inferencing system using the discovered patterns and weight of evidence for classification and prediction； a new way of data retrieval by each class queries for retrieval in a distributive database with unlimited size； supporting validation evidences of the efficacy of the proposed framework and its new development in solving large scaled problem with online interactive capability. 6. Summary and Discussion • N.Y.U.S.T. • I.M.