Research Issues

1. SATIN: Sketch And Transformational INfrastructure Interpreter Interpreter Interpreters Jason I. Hong James Landay Group for User Interface Research UC Berkeley or, Providing Support for Programmers and Designers in Creating Pen-based Applications Why SATIN? Research Issues 1. Many recent research and commercial apps use pens and sketching for interaction. 2. However, much work is re-implemented from scratch. 3. There are few principles - most of it is ad hoc hacks that are not extensible or reusable. The goals of SATIN are to provide a toolkit that allows us to: 1. Simplify system architect and programmer tasks in creating pen-based applications 2. Explore architectures that can be used for a variety of pen-based applications • Key Theme • Do this in a manner that is not a "hack" • Many research and commercial apps have similar themes • These apps often have poor abstractions • These apps often cannot be extended or reused by others • Interface between Recognizers and Interpreters • Recognizers classify ambiguous information • Interpreters act on classifications from Recognizers • Need agreed upon protocols so that Recognizers can be plugged-in (e.g. replace a neural net recognizer with a statistical recognizer) • Dispatching • Correct models for dispatching strokes? • Which GraphicalObject is the target of the stroke? • Who should handle the stroke first? Parent? Target? • How to do progressive interpretation? • For example, the more you draw the more it zooms SATIN System Architecture Sample Usage • 1. GraphicalObjects • GraphicalObjects are objects with state, view(s), and behavior(s) for ink strokes • Add, remove, group, move GraphicalObjects • Rotate, zoom, and semantically zoom • Move objects up and down layers • Properties for line width, translucency, etc • Automatic animation for simple transitions • Automatic rendering and update support • 2. Patch GraphicalObject • Interpreted regions of space • Interpreters help users organize, process, and manipulate sketches (See Sample Usage) • 3. Built-in support for Undo / Redo / Macros • 4. Built-in support for notifications • Notify me when something happens • Used for maintaining constraints and consistency • 5. Multiple Views • Change view based on a property, such as zoom • 6. Multiple Interpreters • Change how ink is handled based on any property • 7. Integration with Java Swing and other apps • Allow use of existing Java widgets • Allow use of Satin as a Java widget Recognize Text • Step A • Step B • Step C Align Bullets Help Sample Applications Runtime Architecture Sheet Sheets bridge Java Swing and Satin. Sheets can be used in Java apps. Java widgets can also be added to Sheets. GraphicalObjects and Interpreters GraphicalObjects are objects with state, behavior(s), and view(s). Behaviors and views can be dynamically defined PC-App (Proof-of-Concept App) DENIM - Quickly sketch out and run web site prototypes Dispatching Java Event Subsystem Stroke Assembler Sheet GraphicalObject Group Java2D Drawing Rendering Subsystem Patch GraphicalObject (GOb) Rendering The Rendering Subsystem simplifies some common Java drawing patterns. Different rendering qualities are used at runtime depending on context. Styles encapsulate color, translucency, and so on, and are automatically used. Commands Command Queue (Undo, Redo, Macros) Command Objects Clipboard Selected GObs Commands Commands implement the Command pattern, encapsulating do, undo, and redo.