Fuzzy Genetics-based Machine Learning Algorithms

1. Fuzzy Genetics-based Machine Learning Algorithms Presented by Vahid Jazayeri

2. outline • Introduction to fuzzy sets and fuzzy rules • Introduction to GBML algorithms • GBML Algorithms • Michigan • Pittsburgh • Hybrid

3. Fuzzy Sets • A Fuzzy set includes a small number of linguistic values • {Small, Large, Medium} • {Small, Medium-small, Medium, Large, Medium-large} • Fuzzy values are assigned to variables instead of exact numerical values • Numerical values should be normalized into [0,1]

4. Cont. • Each number in [0,1] can be each one of fuzzy values but with a degree of membership • Example • 0 is 100% small but 0% large. • 0.125 is 50% small, 50% medium-small, but 0% other values

5. Fuzzy Rules • Answers to classification problems • m classes exist • n variable exists • A fuzzy rule determines a class according to n variable and with a certainty grade • Rjis the j-th rule • Xiis i-th variable • Ajiis the fuzzy value of Xi in the j-th Rule • Cjis the class of the rule Rj • CFjis the certainty factor of the rule Rj

6. Cont. • Don’t-care is also a fuzzy value • Every number is 100% Don’t-care • For shortening the size of rules. • If in a rule Xi is Don’t-care it means we can omit that checking from the rule, so the size of rule is shortened.

7. GBML Algorithms • Genetics-Based Machine Learning algorithms are a simulation of natural genetic phenomena. • Each individual is a string of some meaningful units (characters, bits, …). • A population of individuals generates new populations by the operators: • Crossover • The mixing of two strings and generating two other ones • Mutation • A lowly probable change in some characters of a string • Old, weak individuals are replaced by new, stronger individuals

8. Fuzzy GBML Algorithms • Individuals are fuzzy rules • Fuzzy rules are coded as strings of the following characters: • 1: small • 2: medium-small • 3: medium • 4: medium-large • 5: large • #: don’t-care

9. Cont. • Example: • Says that if x1 is small, x2 is large, x3 is don’t-care and x4 is medium-small • There is no need to encoding the class or certainty grade • They can be calculated dynamically • Each encoded rule has a fitness value determined by a fitness function for discriminating good rules

10. Heuristic Rule Generation Procedure • An encoded rule is prepared. • It is wanted to figure out the class and certainty grade (CF) of the rule Rj according to m experienced predefined training patterns • 1) Calculate the compatibility of each training pattern Xp with Rj: • MAji if the membership function of Aji

11. Cont. • 2) Calculate the sum of compatibility grades for each class • Among c classes of the problem, find the class Cj for Rule Rj that has the maximum value of BClass h(Rj) • The certainty grade of Rj can be calculated as follows:

12. Fuzzy Reasoning • A set of Fuzzy rules are available • A set of normalized values for n variables of a pattern is in hand • It is wanted to classify the pattern • The winner rule of the rule-set is the rule Rj for which: • If Rj is the winner rule then it says that the pattern is in class Cj with the probability of CFj

13. Michigan Algorithm • Generate a set of N fuzzy rules as the initial population of individuals • Evaluate each rule with a fitness function. • Select N/2 pair of individuals • Each pair generate two new individual as the result of crossover • Each newly-born individual may change with a small probability by mutation. Now N new individuals are prepared • Evaluate the new individuals • Replace the best new individuals (with the highest fitness values) with the old bad individuals of the previous population (those with the lowest fitness values)  the new population is now ready • If stopping condition is not satisfied return to step 2

14. Example

15. Evaluation (fitness function) • Set the fitness of each rule to zero • For each training pattern find the winner rule for that pattern. If the class of the winner rule is the same as the class of the pattern, then add one score to the fitness of that rule Else give a negative score to its fitness or do nothing.

16. Pittsburgh Algorithm • The procedure of Pittsburgh algorithm is the same as that of Michigan but with partial differences • Each individual in Michigan is a rule and the population is a collection of rules • Each individual in Pittsburgh is a rule-set and the population is a collection of rule-sets

17. Cont. • The encoding of each rule-set is the concatenation of its encoded rules • Below is an encoded set of 3 rules. Each rule of 3 variables • The fitness of each rule-set is determined by the sum of the fitness of its rules • Final answer in Michigan is the whole population but here is the best rule-set with the highest fitness

18. Comparing two methods

19. Hybrid • The basis of the algorithm is Pittsburgh • A mutation in Pittsburgh means a probable change in an individual, that is a change in a rule-set. A change in a rule-set can be done by executing Michigan on that rule-set, So using Michigan as mutation operator in Pittsburgh is called Hybrid Algorithm • Michigan yields good rules but not necessarily good rule-sets. • Pittsburgh yields good rule-sets but not necessarily good rules. But hybrid algorithm enjoys goods rule-sets with good rules

20. resources • H. Ishibuchi, T. Nakashima, Data Mining techniques, chapter 40, techniques and applications of genetic algorithm-based methods for desiring compact fuzzy classification systems, 1999. • H. Ishibuchi, T. Nakashima, and T. Kuroda, A Hybrid Fuzzy Genetics-based Machine Learning Algorithm: Hybridization of Michigan approach and Pittsburgh approach, 2000.

21. Thanks Any question (?)