Create Presentation
Download Presentation

Download Presentation
## Principled Approximations in Probabilistic Programming

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**The computing stack (approximation)**Algorithms Compiler and runtime Architecture The APPROX view: with probabilitiesand approximations!**The computing stack**Algorithms Compiler and runtime Architecture Probabilistic program Program synthesis Hardware for sampling The APPROX view: with probabilitiesand approximations!**DPMM Clustering**Model • Gaussian Dirichlet Process Mixture Model (DPMM) Cluster 2 New Cluster Gibbs Sampling Algorithm Cluster 3 Cluster 2 Cluster 2 • In each iteration, update the cluster assignments of data points one at a time • For each data point, compute the probability of it belonging to all existing clusters and an unseen cluster • Sample from this probability distribution Cluster 1 Cluster 1 Cluster 1**Sampling in Hardware**35b 1b 32b Comparators Prefix Sum Encoder Probabilities 3b Sample 32b PRNG CLK Idea: exploit errors!**Robustness to hardware faults**Transient Faults Stuck-at Faults Sampler Clustering using DPMM Deka, Biplab. “On Fault Tolerance of Hardware Samplers”. Masters Thesis, University of Illinois at Urbana Champaign, 2014**Approximating compilers**Traditional compiler: • Input: Deterministic program • Goal: Executable semantically equivalent to source • Method: Syntax-guided translation Approximating compiler: • Input: Probabilistic program • Goal: • Satisfy basic boolean invariants • Minimize quantitative error • Method:Program synthesis**Probabilistic programs**heightMan = Gaussian(177,64); heightWoman = Gaussian(164,64); assume(heightWoman > heightMan); return heightMan More complex example: clustering Addition: assertions, angelic nondeterminism Source: Tutorial on Infer.NET by John Winn and Tom Minka**Probabilistic programming ++**• Random variables X: range over distributions • Deterministic variables y • Either holesor temporaries • Functions f(X1,..., Xk, y1,..., yk) • Can map random variables to deterministic ones • Expectation, probability • Assertions • Pareto-optimality goals**Example**X = Gaussian(??, 10); assume (X < 10); c = Pr(X > 0); assert (c > 0.7); minimize (c); Hole One synthesis algorithm in [CCS14] Based on probabilistic abstract interpretation [CCS14] Chaudhuri, Clochard, Solar-Lezama. Bridging boolean and quantitative synthesis using smoothed proof search. POPL 2014.**Use in approximation**Holes = degree of approximation Assertions = invariants, hard bounds Declarative error minimization Deterministic temporaries track resource consumption Synthesis = Compilation