1. Bayesian Density Regression Author: David B. Dunson and Natesh Pillai Presenter: Ya Xue April 28, 2006

2. Outline • Key idea • Proof • Application to HME

3. Bayesian Density Regression with Standard DP • The regression model: (i=1,...,n) • Two cases: Parametric model Standard Dirichlet process mixture model

4. Bayesian Density Regression with Standard DP • Model • The algorithm automatically finds the shrinkage of parameters

5. Polya Urn Model • Standard Polya urn model • This paper proposed a generalized Polya urn model. (1) where is a kernel function. monotonically as increases.

6. Idea – Spatial DP Equation (1) implies • The prior probability of setting decreases as increases. • The prior probability of increases as more neighbors are added that have predictor values xj close to xi. • The expected prior probability of increases in proportion to the hyperparameter .

7. Outline • Key idea • Proof • Application to HME

8. Spatial Varying Regression Model • At a given location in the feature space, A mixture of an innovation random measure and neighboring random measures j~i indexes samples

9. Theorem 1

10. Hierarchical Model • The hierarchical form

11. Conditional Distribution • Let denote an index set for the subjects drawn from the jth mixture component, for j=1,...,n. Then we have for • Conditioning on Z, we can use the Polya urn result to obtain the conditional prior • Only the subvector of elements of belonging to are informative. (2)

12. Marginalize over Z • We obtain the following generalization of the Polya urn scheme (a) (b) if sample i and j belong to the same mixture component.

13. Example For example, n=4, (a) (b) p(mi)

14. Rewrite Equation (2) • Let • Then Eqn.(2) can be expressed as (3)

15. Theorem 4 Hence, Eqn. (3) is equivalent to

16. Predictive distribution

17. Outline • Key idea • Proof • Application to HME

18. Mixture Model • We simulate data from a mixture of two normal linear regression models • Poor results obtained by using the standard DP mixture model.