Chiung Yao Fang Department of Computer Science and Information Engineering

1. An Introduction to Classification Techniques and Their Applications: k-means Clustering and Decision Trees Chiung Yao Fang Department of Computer Science and Information Engineering National Taiwan Normal University Taipei, Taiwan

2. Outline • k-means Clustering • Application: Infant Face Detection • Experimental Results • Decision trees • Application: Infant Facial Expression Recognition • Experimental Results • Conclusion

3. Infant Face Detection • System setup camera

4. Color Space Selection RGB color space HSI color space YCrCb color space H, Cr, Cb model infant skin color

5. An Example

6. k-Means Clustering • Given N input feature dataxt, t=1,...,N • Find k reference vectors which represent input feature data • Reference vectors, mj, j =1,...,k • Use nearest reference • Reconstruction error

8. k-means Clustering

9. Experimental Results original image k = 4 k = 5 k = 6 k = 7 k = 8

10. Experimental Results Input image k=3 k=4 k=5 k=6

11. k-means Clustering • The methods to initial mi • Randomly select k instances • Calculate the mean of all data and add small random vectors • Calculate the principal component partitioning the data into kgroups, and then take the means of these groups • … • How to decide k?

12. Face Region Decision

13. Demo

14. Infant Facial Expression Recognition turn right front gazing laughing crying turn left • Five infant facial expressions • crying, gazing, laughing, yawning and vomiting • Three poses of the infant head • front, turn left and turn right • Total classes: 15 classes vomiting yawning

15. Moments • To calculate three types of moments • Hu moment [Hu1962] • R moment [Liu2008] • Zernike moment [Zhi2008] • Given an image I and let f be an image function. The digital (p, q)th moment of I is given by • The central (p, q)th moments of I can be defined as where and • The normalized central moments of I where

16. Hu moments are translation, scale, and rotation invariant. Hu Moment normalized central moments

17. Example: Hu Moments yawning crying • If the infant facial expressions are different then the values of Hu moments are also different.

18. Liu (2008) proposed ten R moments which can improve the scale invariability of Hu moments. R Moment Hu moments

19. Example: R Moments crying Hu moments • R moments and Hu moments may have different properties.

20. Zernike Moment • Zernike moments of order p with repetition q for an image function f is where To simplify the index, we use Z1, Z2,…, Z10 to represent Z80, Z82,…, Z99, respectively. real part imaginary part

21. Example: Zernike Moments crying

22. Correlation Coefficients A facial expression is a sequential change of the values of the moments. The correlation coefficients of two moments may be used to represent the facial expressions. Let Ai = , i = 1, 2,…, m, indicates the ith moment Ai of the frame Ik, k = 1, 2,…, n. The correlation coefficients between Ai and Aj can be defined as where and : the mean of the elements in Ai

23. The correlation coefficients between seven Hu moment sequences. Correlation Coefficients yawning

24. Decision Trees • Properties of decision trees • An efficient nonparametric method • A hierarchical model • Divided-and-conquer strategy • Supervised learning

25. Tree Uses Nodes, and Leaves

26. Entropy • For node m, Nm instances reach m,Nim belong to class Ci • Node m is pure if pim is 0 or 1 • Measure of impurity is entropy

27. Example of Entropy • In a two-class problem • If p1 = 1 and p2 = 0 all examples are of class C1  the entropy is 0 • If p1 = p2 = 0.5  the entropy is 1 Entropy function for a two-class problem

28. Best Split • If node m is pure, generate a leaf and stop, otherwise split and continue recursively • Impurity after split: Nmj of Nm take branch j, Nimj belong to Ci • Find the variable and split that min impurity

29. A Decision Tree • Decision trees are used to classify the infant facial expressions. H1H3>0 correlation coefficients

30. A Decision Tree The correlation coefficients between two attributesAi and Aj are used to split the training instances. Let the training instances in S be split into two subsets S1 and S2 by the correlation coefficient, then the measure function is The best correlation coefficient selected by the system is

31. Decision tree construction Step 1: Initially, put all the training instances into the root SR, regard SR as an internal decision node and input SRinto a decision node queue. Step 2: Select an internal decision node S from the decision node queue calculate the entropy of node S. If the entropy of node S larger then a threshold Ts, then goto Step 3, else label node S as a leaf node, goto Step 4. Step 3: Find the best correlation coefficient to split the training instances in node S. Split the training instances in S into two nodes S1 and S2 by correlation coefficients and add S1, S2 into the decision node queue. Goto Step 2. Step 4: If the queue is not empty, then goto Step 2, else stop the algorithm.

32. Training: 59 sequences Testing: 30 sequences Five infant facial expressions: crying, laughing, dazing, yawning, vomiting Three different poses of infant head: front, turn left, and turn right Fifteen classes are classified. Experimental Results

33. no yes no yes no yes crying yes no yes no no yes yawning vomiting yawning yes no no crying no yes yes yes no vomiting crying yawning no yes yes no yes no vomiting crying laughing laughing yes no no yes laughing dazing dazing no yes dazing crying Feature type: Hu moments Internal nodes: 16 Leaf nodes: 17 Height: 8

34. Experimental Results • The classification results of the Hu-moment decision tree laughing dazing laughing vomiting

35. Feature type: R moments Internal nodes: 15 Leaf nodes: 17 Height: 10 yes no vomiting yes no yawning yes no yes no yes no dazing laughing crying no yes vomiting no yes yes no yes no crying no yes yes no yes no dazing laughing dazing yes yes no no no yes crying vomiting vomiting crying laughing yes no dazing laughing

36. Experimental Results • The classification results of the R-moment decision tree vomiting yawning dazing dazing

37. yes no yes no yes no yes no yes yes yes no no no crying crying yes no vomiting yes yes no no dazing laughing yes no laughing yawning vomiting yes no no yes yes no yes crying no crying yes crying laughing no yes dazing no yes no yawning dazing dazing yawning vomiting yes no dazing laughing Feature type: Zernike moments Internal nodes: 19 Leaf nodes: 20 Height: 7

38. Experimental Results • The classification results of the Zernike-moment decision tree crying vomiting crying crying

39. Experimental Results • The comparison of the results • The correlation coefficients of the moments are useful attributes to classify the infant facial expressions. • The classification tree created by the Hu moments has less height and number of node, but higher classification rate.

40. Conclusions • Conclusion • k-means Clustering • Example: Infant Face Detection • Decision trees • Example: Infant Facial Expression Recognition