1 / 35

Programming Abstractions for Approximate Computing

Programming Abstractions for Approximate Computing. Michael Carbin with Sasa Misailovic , Hank Hoffmann , Deokhwan Kim, Stelios Sidiroglou , Martin Rinard MIT. Challenges for Programming. Expression (specifying approximations) Reasoning (verifying resulting program)

tehya
Télécharger la présentation

Programming Abstractions for Approximate Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Programming Abstractions for Approximate Computing • Michael Carbin • with SasaMisailovic, Hank Hoffmann, DeokhwanKim, SteliosSidiroglou, Martin Rinard • MIT

  2. Challenges for Programming • Expression (specifying approximations) • Reasoning (verifying resulting program) • Debugging (reproducing failures) • Deployment (changes in assumptions)

  3. Challenges for Programming • Expression (specifying approximations) • Reasoning (verifying resulting program) • Debugging (reproducing failures) • Deployment (changes in assumptions)

  4. Fundamental Questions • Scope • What do we approximate and how? • Specifications of “Correctness” • What is correctness? • Reasoning • How do we reason about correctness?

  5. [Misailovic, Carbin, Achour, Qi, Rinard MIT-TR 14] What do we approximate? • Domains have inherent uncertainty • Execution time dominated by a fraction of code

  6. Approximation Transformations • Approximate HardwarePCMOS, Palem et al. 2005; Narayanan et al., DATE ’10; Liu et al. ASPLOS ’11; Sampson et al, PLDI ’11; Esmaeilzadeh et al. , ASPLOS ’12, MICRO’ 12 • Function Substitution Hoffman et al., APLOS ’11; Ansel et al., CGO ’11; Zhu et al., POPL ‘12 • Approximate Memoization Alvarez et al., IEEE TOC ’05; Chaudhuri et al., FSE ’12; Samadi et al., ASPLOS ’14 • Relaxed Synchronization (Lock Elision) Renganarayanaet al., RACES ’12; Rinard, HotPar ‘13; Misailovic, et al., RACES ’12 • Code PerforationRinard, ICS ‘06; Baek et al., PLDI 10; Misailovic et al., ICSE ’10; Sidiroglou et al., FSE ‘11; Misailovic et al., SAS ‘11; Zhu et al., POPL ‘12; Carbin et al. PEPM ’13;

  7. Example: Loop Perforation floatsum= 0; for (inti = 0; i < n; i += 1) { sum = sum+a[i]; } float avg= sum / n;

  8. Example: Loop Perforation floatsum= 0; for (inti = 0; i < n; i += 1) { sum = sum+a[i]; } float avg= sum / n; float avg = (sum * 2) / n; • Skip iterations (or truncate or random subset) • Potentially add extrapolations to reduce bias i += 2

  9. What is correctness? .c .c Traditional Transformation ≡

  10. What is correctness? .c .c Approximate Transformation

  11. What is correctness? • Safety: satisfies standard unary assertions (type safety, memory safety, partial functionality) • Accuracy: program satisfies relational assertions that constrain difference in results

  12. Relational Assertions • Contribution: program logic and verification system that supports verifying relationships between implementations • x<o>: value of x in original implementation • x<a>: value of x in approximate implementation relate |x<o> - x<a>| / x<o> <= .1; [Carbin, Kim, Misailovic, RinardPLDI’12, PEPM ‘13]

  13. Example: Loop Perforation floatsum= 0; for (inti = 0; i < n; i += 1) { sum = sum+a[i]; } float avg= sum / n; float avg = (sum * 2) / n; i += 2

  14. Example: Loop Perforation • Worst-case error: floatsum= 0; for (inti = 0; i < n; i += 1) { sum = sum+a[i]; } float avg= sum / n; float avg = (sum * 2) / n; i += 2

  15. Novel Safety Verification Concept • Assume validity of assertions in original program • Use relations to prove that approximation does not change validity of assertions • Lowerverification complexity than verifying an assertion outright p<o> == p<a> ∧ safe(*p<o>) safe(*p<a>) assert (safe(*p))

  16. Specifications and Reasoning • Worst-case (for all inputs) • Non-interference [Sampson et al., PLDI ‘11] • Assertions [Carbin et al., PLDI ‘12; PEPM ‘13] • Accuracy [Carbin et al., PLDI ‘12] • Statistical (with some probability) • Assertions [Sampson et al., PLDI ‘14] • Reliability [Carbin, Misailovic, and Rinard, OOPSLA ‘13] • Expected Error [Zhu, Misailovicet al., POPL ‘13] • Probabilistic Error Bounds [Misailovic et al., SAS ‘13]

  17. Questions from Computer Science

  18. Question #1: • “Programmers will never do this.” • - Unnamed Systems and PL Researchers

  19. Challenge Full FunctionalCorrectness Partial Specifications Reasoning Complexity Types Expressivity

  20. Challenge • Ordering of unary assertions still true (improved by relational verification) Full FunctionalCorrectness Partial Specifications Reasoning Complexity Types Expressivity

  21. Challenge Probabilistic Accuracy Bounds Probabilistic Assertions Reliability Worst-case Accuracy Assertions Expected Error Error Distributions Reasoning Complexity Expressivity Performance/Energy Benefit

  22. Example: Loop Perforation(Misailovic et al., SAS ‘11) floatsum= 0; for (inti = 0; i < n; i += 1) { sum = sum+a[i]; } float avg= sum / n; float avg = (sum * 2) / n; i += 2

  23. Example: Loop Perforation(Misailovic et al., SAS ‘11) • Worst-case error: • Probabilistic Error Bound (.95): floatsum= 0; for (inti = 0; i < n; i += 1) { sum = sum+a[i]; } float avg= sum / n; float avg = (sum * 2) / n; i += 2

  24. Question #2: • “There’s no hope for building approximate hardware.” • - Unnamed Computer Architect

  25. Challenge • Approx. PL community must work with hardware community to collaborate on models (not the PL community’s expertise) • Approx. hardware community faces major challenge in publishing deep architectural changes: simulation widely panned • Moving forward may require large joint effort

  26. Question #3: • “I believe your techniques are fundamentally flawed.” • - Unnamed Numerical Analyst

  27. Challenge • Numerical analysts have been wronged • Programming systems have failed to provide support for making the statements they desire • Approximate computing community risks repeating work done by numerical analysis • Opportunity for collaboration • New research opportunities • It’s no longer just about floating-point

  28. Broadly Accessible Motivations • Programming with uncertainty (uncertain operations and data) • Programming unrealizable computation (large scale numerical simulations) • Useful computation from non-digital fabrics(analog, quantum, biological, and human) Opportunity to build reliable and resilient computing systems built upon anything

  29. End

  30. Conclusion • Adoption hinges on tradeoff between expressivity, complexity and benefit • Opportunity/necessity for tighter integration of PL and hardware communities

  31. Open Challenges and Directions • Problem: benefits (performance/energy) may vary between different types of guarantees Probabilistic Accuracy Bounds Worst-case Accuracy Probabilistic Assertions Reasoning Complexity Reliability Error Distributions Expected Accuracy Expressivity

  32. Open Challenges and Directions Expressivity and Reasoning Complexity Type Checking Partial Specifications Full FunctionalCorrectness

  33. Experimental Results

  34. Open Challenges Directions • Traditional Tradeoff Dynamic Analysis Type Checking Partial Specifications Full FunctionalCorrectness Performance/Energy Reasoning Complexity

More Related