Research Issues

1. SATIN: Sketch And Transformational INfrastructure Interpreter Interpreter Interpreters Recognize Text Align Bullets Help Jason I. Hong James Landay Group for User Interface Research UC Berkeley or, Providing Support for Programmers and Designers in Creating Pen-based Applications Why SATIN? Research Issues • Many recent research and commercial apps use pens and sketching for interaction. However, much work is done re-implementing from scratch. Furthermore, there are few principles in the design - most of it is ad hoc. The goals of SATIN are to provide a toolkit that allows us to: • Simplify designer and programmer tasks in creating pen-based applications • Explore architectures that can be used for a variety of pen-based applications • Explore new interaction techniques for using pens and for sketching • Key Theme • How to do all of this in a manner that is not a "hack" • Many research and commercial apps have similar themes and ideas but cannot be extended or reused by others • Event Model • Is the standard windows event model sufficient? • Interface between Recognizers and Interpreters • Recognizers are stateless systems that classify ambiguous information • Interpreters take classifications from Recognizers and act upon them • Need agreed upon protocols so that Recognizers can be replaced by others • Interface for Interpreters • What operations are common for Interpreters? Need to iteratively build them into the system. • Dispatching • How to do progressive interpretation? • For example, the more you draw the more it zooms SATIN System Architecture Sample Usage • Design Goals • 1. Toolkit for supporting the creation of pen-based and sketch-based applications • 2. GraphicalObject manipulation • GraphicalObjects are objects with a view, and dynamically definable state and behavior • Add, remove, group, move, etc • Rotate, zoom, and semantically zoom • Move objects up and down layers • Line width, color, translucency, etc • Automatic animation for smooth transitions • Automatic rendering and update support • 3. Patches • Interpreted regions of space containing GraphicalObjects • Interpreters help users organize, process, and manipulate sketches (See Sample Applications) • 4. Built-in support for Undo / Redo / Macros • 5. Automatic time indexing of operations • 6. Integration with Java Swing and other apps • Standardized Drag and Drop • Export as HTML, GIF, XML • Allow use of existing Java widgets • Allow use of Satin as a Java widget • Step A • Step B • Step C Runtime Architecture Sheet Sheets are the bridge between Java Swing and Satin. Sheets can be added into Java applications. Existing Java widgets can also be used in Satin if they implement the GraphicalObject interface. GraphicalObjects and Interpreters GraphicalObjects are objects with state, behavior, and a graphical view. The state and behavior can be dynamically defined through Interpreters. A GraphicalObject can have one or more Interpreters, assignable at runtime. Sample Applications • PatchBoard • Whiteboard application that lets you draw, manipulate, and interpret drawings • Environment is large interactive screen with Palm and WinCE devices • Denim • Quickly sketch out interfaces or websites • Run them for rapid prototyping and feedback Dispatching Java Event Subsystem Stroke Assembler Sheet GraphicalObject Group Java2D Drawing Rendering Subsystem Patch GraphicalObject (GOb) Rendering The Rendering Subsystem is designed to simplify many of the common patterns encountered when drawing in Java. Styles encapsulate such information as translucency and line width for GraphicalObjects, and are automatically used by the Rendering Subsystem. Commands Command Queue (Undo, Redo, Macros) Command Objects Commands Commands implement the Command pattern, encapsulating do, undo, and redo. Clipboard Selected GObs