Piccolo.NET: A Scalable Structured Graphics Toolkit

1. Piccolo.NET: A Scalable Structured Graphics Toolkit Ben Bederson University of Maryland Human-Computer Interaction Lab Computer Science Department

2. Piccolo.NET:What is it? • Toolkit that supports: • structured canvas of graphical objects • hierarchical scenegraph model • creation of “controls” for use within Windows Forms • Supports 2D object-oriented graphics • hierarchies (transformation, transparency, etc.) • animation, event handling • cameras, layers, views • efficiency mechanisms • Missing structure that relies on underlying renderer • GDI+, Direct3D, OpenGL, Java2D, etc. => Open, Extensible and Efficient Human-Computer Interaction Lab

3. What Is It Good For? • => Structured graphics • Custom interaction • Zoomable User Interfaces Gentleware.com - Poseidon Demo PaperLens – Microsoft Research Human-Computer Interaction Lab

4. What Is It Good For? • Structured graphics • => Custom interaction • Zoomable User Interfaces Demo International Children’s Digital Library www.icdlbooks.org This Presentation Tool Range Slider Human-Computer Interaction Lab

5. What Is It Good For? • Structured graphics • Custom interaction • => Zoomable User Interfaces U. Victoria – Ontology Visualization Demo PhotoMesa www.photomesa.com Human-Computer Interaction Lab

6. Where Does It Run? • Almost everywhere • Piccolo.NET • (C#) .NET for Windows • PocketPiccolo.NET • (C#) Compact Framework for Pocket PC • Piccolo.Java • (Java) for everywhere else Human-Computer Interaction Lab

7. Class Hierarchy Typical run-time structure How Does It Work? • “Monolithic” design • Simple class hierarchy • Simple runtime structure • Utilities for easy startup using UMD.HCIL.Piccolo; using UMD.HCIL.Piccolo.Nodes; using UMD.HCIL.PiccoloX; public class PHelloWorld : PForm { public override void Initialize() { PText text = new PText("Hello World!"); Canvas.Layer.AddChild(text); } static void Main() { new PHelloWorld(); } } Code Human-Computer Interaction Lab

8. What Does It Do? • Let’s look at the “Feature” demos: Demo Human-Computer Interaction Lab

9. Bigger Example: GraphEditor • Nodes and Edges • Highlightable • Draggable • Approach: • Structure of objects • Relate nodes & edges • Event handlers Code Human-Computer Interaction Lab

10. This Presentation Tool • Similar structure to GraphEditor: • Scenegraph • Event handlers • But this time: • Hierarchical transforms • Animation Human-Computer Interaction Lab

11. PocketPiccolo.NET • Same great taste – less filling! • Essentially identical except for: • Lower quality rendering • No graphical paths or transparency • No rotation • Poorer performance • Still all the cool stuff: • Hierarchical, transformable objects • Animation • Event handling • Efficiency mechanisms Demo Human-Computer Interaction Lab

12. History Lesson – ZUI ToolkitsSpatial Data Management System (SDMS): Donelson, MIT • 1978 – SDMS • Tour de force • Ubiquitous computing • ZUIs • Novel input devices • Totally specialized system Human-Computer Interaction Lab

13. History Lesson – ZUI ToolkitsPad: Perlin, NYU • 1993 – Pad • Ran on Sun 2 • Written in C • X Graphics • Jump-zoom, B&W • Impressively fast • Moderately extensible Human-Computer Interaction Lab

14. History Lesson – ZUI ToolkitsPad++: Bederson, UNM • 1994 – Pad++ • Designed for prototyping • Used C++ and Tcl/Tk and X or OpenGL graphics • Smooth zooming • Full color • Bitmaps and vector objects • Didn’t scale up well • The API was defined in Tcl and the C++ code was efficient, but messy… [Bederson & Meyer - SPE 1998] Human-Computer Interaction Lab

15. History Lesson – ZUI ToolkitsJazz: Bederson, UMD • 1998 – Jazz • Wanted ability to build “serious” applications • Moved to Java and object-oriented design • Inspired by 3D graphics: • We built “polylithic” scenegraph • Different than “monolithic” GUI toolkits Human-Computer Interaction Lab

16. Polylithic Potential • Simpler objects, easier to maintain • Decoupled objects, easier to extend • More run-time control • Could better support design environments • But … • More objects to manage • Introduced “editor” to manage object chains • Still not good enough for app programmers Human-Computer Interaction Lab

17. History Lesson – ZUI ToolkitsPiccolo: Bederson, UMD • 2003 – Piccolo • Decided app builder more important than toolkit builder • Added support for C# and Pocket PC • Went back to “monolithic”, but OO design • Finally happy • This is our “last” toolkit • Now have commercial apps being built using Piccolo Human-Computer Interaction Lab

18. How Do Toolkits Compare? • Tough comparison – many design differences aside from polylithic / monolithic architectures • Did case studies and a performance analysis DateLens www.datelens.com DateLens Mockup • Piccolo.dll [120 kb] • PiccoloX.dll [68 kb] Human-Computer Interaction Lab

19. Performance Analysis Scenegraph manipulation speed Rendering Speed [Bederson, Grosjean, Meyer - TSE 2004] Human-Computer Interaction Lab

20. Architectural Reflections • Toolkits are usually worth their weight • Polylithic and Monolithic each have merits • Similar performance and code sizes • Base arch. on expected life cycle • More programmer oriented => monolithic • More design program oriented => polylithic Human-Computer Interaction Lab

21. Language Reflections: Java vs. C# • Languages • C# “usability” advances important for us: properties, events, indexers, foreach, using, attributes • Libraries • Similar, but multiplatform has significant cost • Platform-specific feature use common • Java tortured its APIs with backwards compatibility • IDEs • Eclipse: Free, refactoring support, pluggable, rapid changes • VS.NET: Expensive, but fast, form designer, integrated device emulator, and cheap for academics • Platforms • Multiplatform required for UMD’s required CS courses • But C# is used in some senior/grad courses Demo Human-Computer Interaction Lab

22. Conclusions / Directions • Are toolkits beneficial? • => Clearly yes (sometimes) • Does zooming work? • Is animation helpful? • Good small representations needed • Animation helps maintain object constancy • Understand domain and users • Piccolo’s Future: • Finish website, tutorials, etc. • Hook up to other renderers (Direct3D, OpenGL) • Build more apps! • Funded by: • Microsoft • Initially by: • DARPA • NSF www.cs.umd.edu/hcil/piccolo Human-Computer Interaction Lab