1 / 50

Julian Center on Regression for Proportion Data

Julian Center on Regression for Proportion Data. July 10, 2007 (68). Regression For Proportion Data. Julian Center Creative Research Corp. Andover, MA, USA. Overview. Introduction What is proportion data? What do we mean by regression? Examples Why should you care?

MikeCarlo
Télécharger la présentation

Julian Center on Regression for Proportion Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Julian CenteronRegression for Proportion Data July 10, 2007 (68)

  2. Regression For Proportion Data Julian Center Creative Research Corp. Andover, MA, USA MaxEnt2007

  3. Overview • Introduction • What is proportion data? • What do we mean by regression? • Examples • Why should you care? • Coordinate Transformation to Facilitate Regression. • Measurement Models • Multinomial • Laplace Approximation to Multinomial • Log-Normal • Regression Models • Kernal Regression (Nadaraya-Watson Model) • Gaussian Process Regression • With Log Normal Measurements • With Multinomial Measurements – Expectation Propagation • Conclusion Julian Center

  4. What is Proportion Data? Julian Center

  5. What is Regression? • Regression = Smoothing + Calibration + Interpolation. • Relates data gathered under one set of conditions to data gathered under similar, but different conditions. • Accounts for measurement “noise”. • Determines p(r|x). Julian Center

  6. Examples • Geostatistics: Composition of rock samples at different locations. • Medicine: Response to different levels of treatment. • Political Science: Opinion polls across different demographic groups. • Climate Research: • Infer climate history from fossil pollen samples. • Calibrate model using present day samples from known climates. • Typically, examine 400 pollen grains and sort into 14 categories Julian Center

  7. Why Should You Care? • Either, you have proportion data to analyze. • Or, you want to do pattern classification. • Or, you want to use a similar approach to your problem. • Transform constrained variables so that a Laplace approximation makes sense. • Two different regression techniques. • Expectation Propagation for improving model fit. Julian Center

  8. Coordinate Transformation • Well-known regression methods can’t deal with the pesky constraints of the simplex. • We need a one-to-one mapping between the d-simplex and d-dimensional real vectors. • Then we can model probability distributions on real vectors and relate them to distributions on the simplex. Julian Center

  9. Coordinate Transformation Symmetric Softmax Activation Function Centered Log Ratio Linkage Function The rows of T span the orthogonal Complement of 1(d+1) We can always find T by the Gram-Schmidt Process Julian Center

  10. f Softmax is insensitive to this direction. y2 Simplex ln(y1)=- ln(y2) y1 Coordinate Transformation ln(y2) ln(y1) Image of Simplex Under ln Julian Center

  11. Measurement Models • Multinomial • Log-Normal Julian Center

  12. Measurement Model- Multinomial - Julian Center

  13. Multinomial Measurement Model S=400 R1= Julian Center

  14. Measurement Model- Laplace Approximation - • Some regression methods assume a Gaussian measurement model. • Therefore, we are tempted to approximate each Multinomial measurement with a Gaussian measurement. • Let’s try a Laplace approximation to each measurement. • Laplace Approximation: • Find the peak of the log-likelihood function. • Pick a Gaussian centered at the peak with covariance matrix that matches the negative second derivative of the log-likelihood function at the peak. • Pick an amplitude factor to match the height of the peak. Julian Center

  15. Measurement Model- Laplace Approximation - Julian Center

  16. Laplace Approximation to Multinomial Julian Center

  17. Laplace Approximation to Multinomial Julian Center

  18. Laplace Approximation to Multinomial Julian Center

  19. Laplace Approximation to Multinomial Julian Center

  20. Laplace Approximation to Multinomial Julian Center

  21. Laplace Approximation to Multinomial Julian Center

  22. Measurement Model- Log-Normal - e.g. Over-dispersion or under-dispersion Julian Center

  23. Regression Models • Way of relating data taken under different conditions. • Intuition: Similar conditions should produce similar data. • The best to use methods depends on the problem. • Two methods considered here: • Nadaraya-Watson model. • Gaussian Process model. Julian Center

  24. Nadaraya-Watson Model • Based on applying Parzen density estimation to the joint distribution of f and x Julian Center

  25. All Data Points f x Julian Center

  26. Nadaraya-Watson Model f x Julian Center

  27. Nadaraya-Watson Model Julian Center

  28. Nadaraya Watson Model Julian Center

  29. Nadaraya-Watson Model • Problem: We must compare a new point to every training point. • Solution: • Choose a sparse set of “knots”, and center density components only on knots. • Adjust weights and covariances by “diagnostic training”. • Mixture model training tools apply. Julian Center

  30. Sparse Nadaraya-Watson Model f x Julian Center

  31. Gaussian Process Model • Probability distribution on functions. • Specified by mean function m(x) and covariance kernel k(x1,x2). • For any finite collection of points, the corresponding function values are jointly Gaussian. Julian Center

  32. Gaussian Process Model f x Julian Center

  33. Applying Gaussian Process Regression to Proportion Data • Prior – Model each component of f(x) as a zero-mean Gaussian process with covariance kernel k(x1,x2). Assume that the components of f are independent of each other. • Posterior – Use the Laplace approximations to the measurements and apply Kalman filter methods. • Use Expectation Propagation to improve fit. Julian Center

  34. Sparse Gaussian Process Model Julian Center

  35. Sparse Gaussian Process Model Julian Center

  36. Sparse Gaussian Process Model Julian Center

  37. Sparse Gaussian Process Model Julian Center

  38. GP– Log-Normal Model Julian Center

  39. GP– Log-Normal Model Julian Center

  40. GP – Log-Normal Model 1 1 Julian Center

  41. GP Multinomial Model Julian Center

  42. Expectation Propagation Method Julian Center

  43. Expectation Propagation Method Julian Center

  44. Expectation Propagation Method Julian Center

  45. Expectation Propagation Method Julian Center

  46. Expectation Propagation Method Julian Center

  47. Expectation Propagation Method Julian Center

  48. Choosing the Regression Model If you have two samplings taken under the same conditions, do you want to treat them as coming from a bimodal distribution (NW Model) or combine them into one big sampling (GP Model)? Julian Center

  49. Conclusion • A coordinate transformation makes it possible to analyze proportion data with known regression methods. • The Multinomial distribution can be well approximated by a Gaussian on the transformed variable. • The choice of regression model depends on the effect that you want – multimodal vs unimodal fit. Julian Center

  50. Thank you! Julian Center

More Related