1 / 21

Kernel Machines and Additive Fuzzy Systems: Classification and Function Approximation

This paper explores the intersection of kernel machines and additive fuzzy systems, focusing on their applications in classification and function approximation. The authors, Yixin Chen and James Z. Wang from Pennsylvania State University, discuss key concepts including VC theory, Support Vector Machines (SVM), and the construction of fuzzy systems. They present experimental results that demonstrate the effectiveness of their approach using the USPS dataset, along with insights on model complexity, parameter estimation, and generalization performance. Future work and conclusions are also outlined.

naomi-buckner
Télécharger la présentation

Kernel Machines and Additive Fuzzy Systems: Classification and Function Approximation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Kernel Machines and Additive Fuzzy Systems:Classification and Function Approximation Yixin Chen and James Z. Wang The Pennsylvania State University http://www.cse.psu.edu/~yixchen FUZZ-IEEE 2003

  2. Outline • Introduction • VC theory and Support Vector Machines • Additive fuzzy systems and kernel machines • Support vector learning for a class of additive fuzzy systems • Experimental results • Conclusions and future work FUZZ-IEEE 2003

  3. Introduction • Building a fuzzy system • Structure identification • Parameter estimation • Model validation • Do we get a good fuzzy model? • How capable can a fuzzy model be? • How well can the model generalize? FUZZ-IEEE 2003

  4. Introduction (continued) • Several types of fuzzy models are “universal approximators” • Generalization performance • Structural risk minimization • Bias variance dilemma • Overfitting phenomena A “right” tradeoff between training accuracy and model complexity FUZZ-IEEE 2003

  5. Introduction (continued) • Two approaches to find a “right” tradeoff • Cross-validation for model selection • Model reduction to simplify the model • Vapnik-Chervonenkis (VC) theory • A general measure of model set complexity • Bounds on generalization • Support Vector Machines (SVM) FUZZ-IEEE 2003

  6. Outline • Introduction • VC theory and Support Vector Machines • Additive fuzzy systems and kernel machines • Support vector learning for a class of additive fuzzy systems • Experimental results • Conclusions and future work FUZZ-IEEE 2003

  7. VC Theory and Support Vector Machines • One result from VC Theory Binary classification: given a set of training samples drawn independently from some unknown distribution , with probability , the probability of misclassification for any decision function is bounded above by FUZZ-IEEE 2003

  8. VC Theory and Support Vector Machines (continued) • Support Vector Machines (SVMs) • Optimal separating hyperplane FUZZ-IEEE 2003

  9. VC Theory and Support Vector Machines (continued) • Kernel trick A Mercer kernel is a function, , satisfying where is sometimes referred to as the Mercer features FUZZ-IEEE 2003

  10. VC Theory and Support Vector Machines (continued) • Quadratic programming • Decision function FUZZ-IEEE 2003

  11. Outline • Introduction • VC theory and Support Vector Machines • Additive fuzzy systems and kernel machines • Support vector learning for a class of additive fuzzy systems • Experimental results • Conclusions and future work FUZZ-IEEE 2003

  12. Additive Fuzzy Systems and Kernel Machines • Kernel Machines • A class of additive fuzzy systems is functionally equivalent to a class of kernel machines FUZZ-IEEE 2003

  13. Additive Fuzzy Systems and Kernel Machines (continued) • Additive Fuzzy System (AFS) • mfuzzy rules of the form • Product as fuzzy conjunction operator • Addition for fuzzy rule aggregation • First order moment defuzzification • Reference function • Kernel is the product of reference functions FUZZ-IEEE 2003

  14. Additive Fuzzy Systems and Kernel Machines (continued) • Positive definite fuzzy systems (PDFS) • Reference functions are positive definite functions Mercer kernels Examples: Gaussian Symmetric triangle Cauchy Hyperbolic secant Laplace Squared sinc FUZZ-IEEE 2003

  15. Outline • Introduction • VC theory and Support Vector Machines • Additive fuzzy systems and kernel machines • Support vector learning for a class of additive fuzzy systems • Experimental results • Conclusions and future work FUZZ-IEEE 2003

  16. Support Vector Learning for a Class of Additive Fuzzy Systems SVM PDFS Kernel Reference functions Support vectors IF-part of fuzzy rules Lagrange multiplier THEN-part of fuzzy rules FUZZ-IEEE 2003

  17. Outline • Introduction • VC theory and Support Vector Machines • Additive fuzzy systems and kernel machines • Support vector learning for a class of additive fuzzy systems • Experimental results • Conclusions and future work FUZZ-IEEE 2003

  18. Experimental Results • USPS data set • Training data (7291), testing data (2007) • 5-fold cross-validation to determine parameters FUZZ-IEEE 2003

  19. Experimental Results (continued) Linear SVM: 91.3% k-nearest neighbor: 94.3% FUZZ-IEEE 2003

  20. Outline • Introduction • VC theory and Support Vector Machines • Additive fuzzy systems and kernel machines • Support vector learning for a class of additive fuzzy systems • Experimental results • Conclusions and future work FUZZ-IEEE 2003

  21. Conclusions and Future Work Fuzzy Systems Kernel Machines PDFS FUZZ-IEEE 2003

More Related