A cursory glance at machine learning

1. A cursory glance at machine learning Ashwath Rajan

2. Overview, in brief • Marriage between statistics, linear algebra, calculus, and computer science • Machine Learning: • Supervised Learning • ex: linear Regression • Unsupervised Learning • ex: clustering • Discriminative Methods • Learns borders in the feature space; border chosen to minimize error functions • Generative Methods • Learns probabilistic distribution of each class, dependent on parameters; gives confidence of classification

3. Supervised learninglinear regression • Regression can be used to fit a numerical model to a parameter space • To solve, minimize error function • Can solve analytically, or iteratively Test Set Predictions

4. Linear regression

5. Iterative linear regression Turns out to be )

6. Supervised learningsupport vector machines • Mechanism for both pattern recognition and regression • Finds maximum margin n-1 dimensional hyperplanes that split n dimensional parameter spaces • Data can be separable or non-separable Separable vs Non-separable data sets

7. Unsupervised learningk-means clustering Procedure – 0. Dictate number of clusters (k) Randomly select k starting points. These serve as the initial group centroids. Associate the remaining points with the nearest centroid. Move the centroids to the center of their respective clusters Repeat steps 2 and 3

8. Unsupervised learning • If number of clusters is unknown, can use different algorithms, where instead of setting # means, we set size of relative neighborhood

9. Frequentist vs. Bayesian • Maximum Likelihood • Assumes fixed value for parameters • Can be used analytically • Bayesian estimation • Assumes parameter as distribution • Uses evidence to amend prior distribution into posterior

10. Probability and Bayes rule • Bayesian probability estimates allow prior distributions to be modified with discovered data.

11. Bayes rule – cancer example Say, probability of rare cancer: • Probability of no cancer: Now say, there is a blood test to detect cancer • Its fairly accurate, as described by the following table: Sensitivity Specificity

12. Bayes rule – cancer example • So, what happens if you get a positive result? • What is the chance you have cancer? Use Bayes Rule: Sensitivity = .8 * .01 / .594 = .13 Specificity

13. My research at USC • Machine Learning to help Smart Grid • Take building sensor data, and find models to connect different data streams to kWh usage • Both supervised and unsupervised techniques could be considered – however, supervised learning is often most apt

14. Online courses • Much of this material has been shamelessly reproduced/copied from online coursework: • Udacity: Statistics 101 and CS 373 AI • Coursera: Great 10 week machine learning course https://www.coursera.org/course/ml

15. Cited • Regression Example: Andrew Ng, Stanford – Coursera • Cancer Example: Sebastian Thrum, Udacity • “A Tutorial on Support Vector Machines for Pattern Recognition” - CHRISTOPHER J.C. BURGES; Bell Laboratories, Lucent Technologies 1998 • Pattern Recognition Primer – David Doria, 2008