programming by example n.
Skip this Video
Loading SlideShow in 5 Seconds..
Programming by Example PowerPoint Presentation
Download Presentation
Programming by Example

Programming by Example

154 Vues Download Presentation
Télécharger la présentation

Programming by Example

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Programming by Example • Henry Lieberman • Media Laboratory • Massachusetts Institute of Technology • Cambridge, MA, USA • • • PBE Web site: /PBE/

  2. Your Wish is My Command:Programming by Example • Morgan Kaufmann, 2001

  3. Programming should be like teaching • User plays the role of a teacher • Computer plays the role of a student

  4. Programming by Example • (Also called "Programming by Demonstration") • Agent "watches what you do" in an interactive interface • Records sequence of operations, data • Generalizes program so that you can use an analogous procedure in new examples • "Macros on Steroids"

  5. Programming in the User Interface • The same interface used by end-users can be used for programming • Empowers end-users to do programming • No switching between user and programmer interfaces • Empowers "visual thinkers"

  6. PBE and the "Intelligent Agent" movement • There's been a recent movement toward "intelligent agents" -- proactive, assistive, (possibly anthropomorphic) interfaces • PBE can be used to make agents instructible • Agent personalities: • Butler/Servant/Secretary • Teacher/Student, Master/Apprentice

  7. Issues in Programming by Example • The Generalization Problem (= Data Description) • How do I describe actions and data? • How similar do the actions/data have to be next time? • Feedback • How does the system show the user what it's learned? • Advice • How can the user influence or modify what the system learns?

  8. Issues in Programming by Example • How should we record the user's actions? • At what granularity should things be recorded? • Can the user interact with the agent during recording? If so, how? • Do we represent the agent directly in the interface? • Do we anthropomorphize the agent? • Can users give advice during recording? • Turning recording on and off

  9. Issues in Programming by Example • How should the recorded program be represented? • Should the user see the recorded program? • What should it look like? • Should the user interact directly with the recorded program? If so, how? • How to represent examples? • How to represent generalizations?

  10. Issues in Programming by Example • How should the user edit an already-recorded program? • How do we keep the representations consistent during editing? • Can the user edit previous generalizations? • How can users debug programs? • Can users step/trace/break programs?

  11. Machine learning in Programming by Example • Lots of work in machine learning • Careful: Watch for hidden assumptions • Few examples, learns fast, explainable • Explanation-based Generalization • Case Based reasoning • Lots of examples, slow but more robust • Neural networks, PAC learning, data-mining

  12. Users of the world, unite! • You have nothing to lose but your error messages! • Almost "political" message of empowerment of users • Users should not be at the mercy of software providers • User procedures and data should not be held hostage by software companies

  13. "Poor man's"Programming by Example • Macro recorders in applications, AppleScript • Emacs keyboard macros • Interface builders / Scripting languages • Quickeys / OneClick …

  14. Hooking PBE systems to conventional applications • PBE systems would like to use conventional interfaces as tools rather than re-implement apps • Conventional interfaces aren't set up to be operated by an external program • Issues: Access to application data, sharing the interface, parallelism • Article: Lieberman, IUI 98 "Attaching Interface Agents to Conventional Applications"

  15. Watch What I Do • Allen Cypher, ed. MIT Press 1993

  16. A Small Matter of Programming • Bonnie Nardi, MIT Press, 1994. • End-User Programming • Scripting languages • Application-specific languages • "Gardeners"

  17. Eager - Allen Cypher • Programming by example for Hypercard • Notices patterns in user actions • Anticipation highlighting of predicted actions • Anthropomorphization of learning agent • Example: Collect subject lines from messages

  18. Eager

  19. Mondrian: An Instructible Graphical Editor • Mondrian = • An object-oriented graphical editor + • An agent that records user interface actions + • Learning and generalization heuristics

  20. Mondrian: An Instructible Graphical Editor

  21. Wizard of Oz Experiment • Human simulates agent • What kinds of communication between human and agent are plausible? • What kind of knowledge does the agent need to know to perform the task?

  22. Wizard of Oz experiment task • Reformat a Bibliography • Lieberman, H., The One True Path to Intelligent Agents, International Conference on Buzzwords and Hype, Tahiti, 1996. • [Lieberman 96] The One True Path to Intelligent Agents, Int. Conf. on Buzzwords and Hype, Tahiti, (1996). • Oz, Wizard Of, Pay No Attention to the Man Behind the Curtain, Journal of Smoke and Mirrors, August 1942, Emerald City, Oz.

  23. Wizard of Oz results • Users can make mistakes • Information can be incomplete • Agents can ask the user if they encounter an error situation • Agent could ask the user to supply missing information • Agent could ask the user to edit the example • Skip steps that don't work

  24. Wizard of Oz experiment method • User performs task once • “Thinking aloud”: goals and methods • Agent (secretly) constructs description of actions • In terms of simple “implementable” operations • Agent offers help • Agent performs help according to action description • Agent gives user feedback • Agent accepts advice from user

  25. Survey of "Your Wish" systems • Astonishingly wide variety of application domains: • Kids, Games, CAD, GIS, Text editing, Graphical editing, Web browsing, Animation, Databases, etc. • Illustrates variety of design decisions in PBE systems • Different reps for programs, agents, etc. • How much inference, generalization, initiative?

  26. Stagecast Creator - Smith, Cypher • Programming for kids, grid game world • If you see this pattern of squares, replace it with that pattern of squares • Viz rep of rules, editor/stepper • Generalization on object types, introduce variables • Commercially available, success with thousands of users

  27. A Stagecast Creator Rule

  28. A Stagecast Creator Game

  29. Toontalk - Ken Kahn • Programming language is a video game • Most radical programming model • Concurrent logic programming • Abstraction by removing detail • Commercially available, success with kids in schools

  30. Toontalk's Visual Language

  31. Generalizing by Removing Detail

  32. Toontalk Sort program

  33. How much intelligence?- Brad Myers, Rich McDaniel • CMU group one of most active and consistent in PBE • Multitude of systems, "gems" of PBE • Peridot, Lapidary, Jade, Gilt, Tourmaline, C32, Pursuit, Gold, Marquise, Katie, Turquoise, Topaz, Gamut • Experience with range of conservative to radical PBE systems • "Spectrum of Instructibility" • Tradeoff between conveience and control

  34. Pursuit's Visual Language

  35. Selecting Generalizations in Marquise

  36. Internet Scrapbook - Sugiura • Everybody's favorite application area - the Web! • Automates Browse & Cut & Paste applications • How do you describe a Web page selection? • Partial matching, Web design heuristics • Commercially available - preloaded on NEC PCs

  37. Internet Scrapbook

  38. TriAS - Bauer, Meyer, Dengler, Paul • Automating Browse & Copy & Paste, like Scrapbook • Collaborative, mixed-initiative dialog • At any time, agent or user can propose selections, generalizations • What Next? Can I Stop? • Integrates wrapper description language HyQL, InfoBeans dynamic Web sources

  39. Collaborative Wrapper Definition

  40. PBE for GIS: CSprl - Traynor, Williams • Study of how end users use Geographical Information Systems • "Gardeners", "Surrogate Users" • Comic-strip PBE language

  41. A Geographical Info Systems PBE Program

  42. Visual Language for Geographic Information Systems

  43. PBE for Computer Aided Design -Girard, Pierra • True "Industrial Strength" PBE, probably the champ • CAD users are sophisticated in application domain, willing to specify a lot • Provides complete editing and debugging • Key issues: Naming, Ambiguity in spec/solutions, general control structures: conditionals, loops

  44. Example-Based Programming in Computer-Aided Design

  45. Demonstrating "Hidden Features" - McDaniel, Myers • PBE for animated computer games • WYSIWYG - "What you get is all you have" • Demonstrate "hidden features" by reifying them • Guides, e.g. Arrows for directions • Sophisticated heuristics, lots of inference • Stop This! • Do Something!

  46. Demonstrating Hidden Features

  47. Report Generation - Masuishi • Mundane, but hey, could save a lot of time • Format generalization • Application-specific iteration • Distributed to wide user community

  48. Report Generation by Example

  49. Dynamic Macro - Masui • Notices loops in text editing commands • Simple, automatic: No start/stop recording • Wide community of users • Also: POBox predictive input for PDAs, phones, etc.

  50. Dynamic Macro