Pattern Recognition Systems

1. FuzzySystems ToolBox, Mark Beale and Howard Demuth Pattern Recognition Systems • One field is concerned with systems that recognize patterns with a classification scheme that has already been determined. • Example: Character recognition, the categories (26 letter of alphabet) are already known. • Referred to as pattern classification.

2. FuzzySystems ToolBox, Mark Beale and Howard Demuth Pattern Recognition Systems • Other field is concerned with obtaining a classifier for a set of patterns where the categories are not initially known. • Example: Clustering data concerning patients to see whether patients can be grouped in a way that helps indicate diagnoses. • Referred to as Pattern clustering

3. FuzzySystems ToolBox, Mark Beale and Howard Demuth Pattern Classification • Attempt to pick the correct category when presented with a pattern. • Solution is very problem specific • Character recognition will be different from recognizing the tank in the battle field

4. FuzzySystems ToolBox, Mark Beale and Howard Demuth Design a Fuzzy Pattern Classifier • Pattern Features • Create fuzzy sets that represent features in the data that are considered important. • Example: ratio of the character’s height to its width may be important while the size of a character is not!

5. FuzzySystems ToolBox, Mark Beale and Howard Demuth Design a Fuzzy Pattern Classifier • Defining the System • fuzzy sets are used to convert crisp inputs to grades that are then combined by a pattern classifier to obtain grades for each of the possible classification categories. • Highly problem dependent!

6. FuzzySystems ToolBox, Mark Beale and Howard Demuth Design a Fuzzy Pattern Classifier • Testing the System • use known and unknown test sets. • Typically, the system will not perform as well as desired at first. • One may have to test a fuzzy classifier • find why it gave a wrong result, • make changes to improve the classifier.

7. FuzzySystems ToolBox, Mark Beale and Howard Demuth Application:Character Recognition • Goal: design a system that will pick the correct character when it is given a bit map representation of a printed or written input character. • Example: classifying the ten digits o through 9.

8. FuzzySystems ToolBox, Mark Beale and Howard Demuth Pattern Features • INPUT • bit map • 25 horizontal X 37 vertical pixels (925 pixel field) • each pixel have a 0 or 1 associated with it.

9. FuzzySystems ToolBox, Mark Beale and Howard Demuth Pattern Features • Creating a rule set dealing with 925 inputs would be extremely difficult and time consuming. • Fuzzify the input, resulting in a compact summary of the overall state of the bit map.

10. FuzzySystems ToolBox, Mark Beale and Howard Demuth Fuzzy sets for the Horizontal Position • Define five triangular-shaped fuzzy sets for the horizontal position. • xS = 0:24; • xL = eqshape(3,xS,5); • xG = shapeval(xL, xS);

11. FuzzySystems ToolBox, Mark Beale and Howard Demuth Fuzzy sets for the Vertical Position • Define seven triangular-shaped fuzzy sets for the vertical position. • yS = 0:36; • yL = eqshape(3,yS,7); • yG = shapeval(yL, yS);

12. FuzzySystems ToolBox, Mark Beale and Howard Demuth Fuzzify the input pixel • Given any pixel (horizontal x and verticle y), the membership of that pixel can be calculated for each of 5 horizontal and 7 vertical regions. • G1 = fuzzyval (xS,xG,x); • G2 = fuzzyval (yS,yG,x);

13. FuzzySystems ToolBox, Mark Beale and Howard Demuth Crate 5 x 7 regions of the 2-D bitmap • Find all combinations of intersections between the two groups of fuzzy sets • G3 = allcomb(‘min’, G1, g2); • G3 wil be a 35-element column vector of grades associated with the 35 regions of the 2-D bit map

14. FuzzySystems ToolBox, Mark Beale and Howard Demuth Multiple Pixels • If vectors X and Y represent the input pixels, then • G1 = fuzzyval(xS, xG, X); • G2 = fuzzyval(yS,yG, Y); and • G3 = allcomb(‘min’, G1, G2); • G3 represents a 35 -element column for every pixel crossed by all of the input pixels

15. FuzzySystems ToolBox, Mark Beale and Howard Demuth Single Grade for each 5 x 7 regions • Taking the union of all grades associated with each square region: • G = or (G3 ’ ) ’; • We now have 35 grades, each grade measuring to what extent the user’s drawn character passed through a different region. • 925-pixel bit map ----> 35 grades!

16. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • Once the good features have been found, one can define a system to transform the feature grades into the grades for each class.

17. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • Zero: • G0 = and([ or( G([2 7])); or( G([3 8])); or( G([4 9])); • or( G([11 12])); or( G([16 17])); or( G([21 22])); • or( G([14 15])); or( G([19 20])); or( G([24 25])); • or( G([27 32])); or( G([28 33])); or( G([29 34])); • not( G([13 18 23]) ) ]);

18. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • One:

19. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • One: • G1 = or([ and([ G([5 9 18 27 31]); • not(G([1 2 3 6 7 11 25 29 30 33 34 35])) ]); • and([ G([1 7 18 29 35]); • not(G([3 4 5 9 10 15 21 26 27 31 32 33])) ]); • and([ G([1 6 11 16 21 26 31]); • not(G([3 4 5 8 10 13 15 18 20 23 25 ... • 28 30 33 34 35])) ]); • and([ G([2 3 4 5]); • not(G([6 7 10 11 12 15 16 17 20 25 30 35])); • or( G([8 9])); • or( G([13 14])); • or( G([18 19])); • or( G([23 24])); • or( G([28 29])); • or( G([33 34])); ]) ]);

20. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • Two: • G2 = and([ somewhat(G(2)); • G([3 32 33 34]); • or( G([6 7])); or( G([30 25])); or( G([4 5])); • not([ G([22 28]); and(G([10 15])) ]) ]);

21. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • Three: • G3 = and([ G([2 3 19 32 33 34]); • or( G([9 10]) ); • or( G([14 15]) ); • not( G([5 16 21]) ) ]);

22. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • Four: • G4 = and([ or([ and(G([4 14 19 29])); and(G([5 15 20 30])) • and(G([3 13 18 28])) ]); • or([and(G([12 14 15])); and(G([17 19 20])); • and(G([22 24])) ]); • not([ G([1 2 6 7]); and(G([32 33 34])) ]) ]);

23. FuzzySystems ToolBox, Mark Beale and Howard Demuth Defining the System • Once the grades for each class have been found, they can be combined into a single grade vector, cG • cG = [G0 G1 G2 G3 G4 G5 G6 G7 G8 G9]; • THEN THE MAXIMUMGRADE CHARACTERS CAN BE FOUND • C = cS(find(cG >= max(cG)));

24. FuzzySystems ToolBox, Mark Beale and Howard Demuth Testinging the System • Demo • Matlab : numrec

25. FuzzyLogic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall Pattern Clustering • Unsupervised Clustering • Find interesting patterns or groupings in a given set of data. • Example: perform the task of “segmenting” the images (I.e., partitioning pixels on an image into regions that correspond to different objects or different faces of objects in the images).

26. FuzzyLogic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall Pattern Clustering • Conventional clustering algorithms find a “Hard partition” of a given dataset based on certain criteria that evaluate the goodness of a partition. • “Hard Partition” - each datum belongs to exactly one cluster of the partition. • In many real-world clustering problems, some data points partially belong to multiple clusters (eg. A pixel in a MRI may correspond to a mixture of two different types of tissues.

27. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Soft Clustering Algorithms • Find a “soft partition” of a given data set based on certain criteria. • A datum can partially belong to multiple clusters • In practice, most soft clustering algorithms do generate a soft partition that also forms a fuzzy partition.

28. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means Algorithm • Best known fuzzy clustering algorithm • produces a constrained soft partition • I.e. • Smcj(xi) = 1 • Both fuzzy c-means and probabilistic clustering produce a partition of similar properties, the clustering criteria underlying these algorithms are very different!

29. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means Algorithm • Generalizes a hard clustering algorithm called the c-means algorithm • Hard C-Means algorithm (HCM) aims to identify compact well-separated clusters • compact cluster has a “ball like” shape • the center of the ball is called the center or the prototype of the cluster.

30. Example of two clusters that are not compact and well separated FuzzyLogic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall

31. Example of two clusters that are compact, but not well separated FuzzyLogic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall

32. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means Algorithm • Like HCM, fuzzy c-means also tries to find a good partition by searching for prototypes vi that minimize the objective function Jm • k=1,2,…c • m is a weight that affects how partial members of a cluster affect the clustering result • FCM also needs to search for membership function mci that minimize Jm

33. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means Theorem • A constrainted fuzzy partition {C1, C2, …, Ck} can be a local minimum of the objective function Jm only if the following conditions are satisfied: • Based on this theorem, FCM updates the prototypes and the membership function iteratively using Equation A and B until a convergence criterion is reached

34. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means Algorithm

35. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means - Example • Use FCM to partition the data set into two clusters (c=2), suppose we set the parameter m in FCM at 2, and initial prototypes to v1 = (5,5) v2 = (10,10)

36. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means - Example

37. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Fuzzy C-Means - Example • The initial membership functions of the two clusters are calculated using Equation A

38. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Example (Continued) • Similarly,

39. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Example (Continued) • Similarly,

40. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Example (Continued) • FCM algoruthm then updates the prototypes according to Equation B • For V1

41. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Example (Continued) • FCM algoruthm then updates the prototypes according to Equation B • For V2

42. FuzzyLogic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall Example (Continued) • The updated prototype v1 and v2 are move closer to the center of the cluster formed by x1,x2, x3 and x4,x5,x6 , respectively • Show Matlab example

43. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes • Used in PDAs • Interactive CAD systems

44. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Multi-stroke shapes

45. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Multi-stroke shapes

46. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Multi-stroke shapes

47. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Uni-stroke shapes

48. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Uni-stroke shapes

49. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Recognition algorithm: 1. Use global geometric properties of the shapes 2. Use a set of filters to either to identify shapes or to filter out unwanted outcomes 3. Use fuzzy logic to handle uncertainty

50. Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively Recognition of Geometric shapes Feature extraction Use statistical properties of shape’s features!