310 likes | 460 Vues
Convex Optimization in Machine Learning. MURI Meeting July 2002 Gert Lanckriet ( gert@eecs.berkeley.edu ) L. El Ghaoui, M. Jordan, C. Bhattacharrya, N. Cristianini, P. Bartlett U.C. Berkeley. Convex Optimization in Machine Learning. Advanced Convex Optimization in Machine Learning. SDP.
E N D
Convex Optimization in Machine Learning MURI MeetingJuly 2002 Gert Lanckriet (gert@eecs.berkeley.edu) L. El Ghaoui, M. Jordan, C. Bhattacharrya, N. Cristianini, P. Bartlett U.C. Berkeley
Advanced Convex Optimization in Machine Learning SDP SOCP QCQP QP LP
MPM: Problem Sketch (1) aT z = b : decision hyperplane
Probability of misclassification… … should be minimized ! … for worst-case class-conditional density… MPM: Problem Sketch (3)
MPM: Geometric Interpretation
Robustness to Estimation Errors: Robust MPM (R-MPM)
MPM: Convex Optimization to solve the problem Lemma Linear Classifier Convex Optimization: Second Order Cone Program (SOCP) Kernelizing Nonlinear Classifier ) competitive with Quadratic Program (QP) SVMs
MPM: Empirical results a=1–b and TSA (test-set accuracy) of the MPM, compared to BPB (best performance in Breiman's report (Arcing classifiers, 1996)) and SVMs. (averages for 50 random partitions into 90% training and 10% test sets) • Comparable with existing literature, SVMs • a=1-b is indeed smaller than the test-set accuracy in all cases (consistent with b as worst-case bound on probability of misclassification) • Kernelizing leads to more powerfull decision boundaries (alinear decision boundary < anonlinear decision boundary (Gaussian kernel))
Machine learning Kernel-based machine learning The idea (1)
training set (labelled) test set (unlabelled) The idea (4)
Hard margin SVM classifiers (7) training set (labelled) test set (unlabelled) Learning the kernel matrix !
Hard margin SVM classifiers (11) Learning Kernel Matrix with SDP !