A New Approach to Exploring Information on the Web

1. VTL05 A New Approach to Exploring Information on the Web Gary William Flake, Ph.D. Technical Fellow Live Labs

2. Preface

3. A Reintroduction to Live Labs • You may already know our work (Seadragon, Photosynth, Deepfish, …) • Everything we do is an experiment: • Some experiments fail (but we learn) • Some experiments succeed (and make our products better) • Today: our most ambitious experiment ever • You can take part in the experiment

4. Framing this Presentation • Challenges • The Web is growing in complexity and size • Users view dozens of items for but one insight • Data and user models not always aligned • Opportunities • Treat the Web more holistically • Create an interaction model designed for the modern internet • Focus more on the data and less on the pages where the data resides

5. Introducing Pivot • Pivot is fundamentally about: • Visualizing collections of objects • Revealing hidden patterns within data • Acting on discovered insights • You’ll probably compare Pivot to: • File explorers> Search engines • Web browsers>Spreadsheets, or DBs • We think that Pivot suggests an entirely new application category

6. Demo

7. A Quick Thank You To our Content Partners

8. How Did We Do It? A Technical Introduction

9. Technical Introduction • Basic application architecture • Backend basics: • Deep Zoom items • Deep Zoom collections • Collection XML (CXML) • Backend design patterns: • Simple collections • Linked collections

10. Application Architecture Rich Local Client Back End Service File, Web, and/or DB Server Request … DZI DZI DZC as HTTP GET .Net Client Seadragon Trident / IE Response CXML DB as XML and images

11. Technical Introduction • Basic application architecture • Backend basics: • Deep Zoom items • Deep Zoom collections • Collection XML (CXML) • Backend design patterns: • Simple collections • Linked collections

12. Deep Zoom Items (DZI)Pyramid, levels, tiles, images • Each tile is an image file • Each level is a set of image files in a folder • Each pyramid is a set of folders with image tiles for each level • A simple XML file describes the size of the image • Images can have only some tiles (sparse images) • Pyramid 1.4x original size • 10Mpixels converts in 0.5s

13. Deep Zoom Items (DZI)XML manifest • <?xml version="1.0" encoding="UTF-8"?> • <Image TileSize="254" Overlap="1" Format="jpg" xmlns="http://schemas.microsoft .com/deepzoom/2008"> • <Size Width="7530" Height="9155"/> • </Image>

14. Deep Zoom Collections (DZC)Pyramid, levels, tiles, files • Thumbnails are packed into shared tiles • Loading 100’s of images requires loading few tiles • Patented Morton layout makes adding items O(1) • No more than one tile per level is not full • Optimizes rendering by sharing textures • Storage is roughly one thumbnail per image

15. Deep Zoom Collections (DZC)XML manifest • <?xml version="1.0" encoding="UTF-8"?> • <Collection MaxLevel="8" TileSize="256“ Format="jpg“ • Quality="0.95" NextItemId="5" xmlns="http://schemas.Microsoft.com/deepzoom/2008"> • <Items> • <I Id="0" N="0" IsPath="1" Source="NA_images/Const1.dzi"> <Size Width="7258" Height="8785"/> • </I> • <I Id="1" N="1" IsPath="1“ Source="NA_images/Const2.dzi"> • <Size Width="7330" Height="8904"/> • </I> • ... • </Items> • </Collection>

16. Collection XML (CXML)Facets as property / value pairs • Enumerates facet category / facet value pairings mapped to items • Can contain detailed information for each item (for info pane) • Can be split in parts to allow for incremental loading Facet Categories Facets Items

17. Collection XML (CXML)Preamble, facet categories, and closing • <?xml version="1.0" encoding="utf-8"?> • <Collection Name="People Collection" SchemaVersion="1" • xmlns="http://schemas.microsoft.com/livelabs • /pivot/collection/2009"> • <FacetCategories> • <FacetCategory Name="Nickname" Type="String" /> • <FacetCategory Name="Weight" Type="Number" /> • <FacetCategory Name="Birthday" Type="DateTime" /> • </FacetCategories> • <Items ImgBase="SomePeople.dzc"> • ... • </Items> • </Collection>

18. Collection XML (CXML)Items • <Item Img="#0" Id="0“ Name="Alice“ • Href="http://fabrikam.com/person/Alice"> • <Description>This is a description of Alice</Description> • <Facets> • <Facet Name="Nickname"><String Value="Ali" /></Facet> • <Facet Name="Weight"><Number Value="123" /></Facet> • <Facet Name="Birthday"> • <DateTime Value="1982-02-10T14:51:00" /> • </Facet> • </Facets> • </Item> • ...

19. Technical Introduction • Basic application architecture • Backend basics: • Deep Zoom items • Deep Zoom collections • Collection XML (CXML) • Backend design patterns: • Simple collections • Linked collections

20. Simple Collections • Up to 5,000 items • Metadata is static and loaded all at once • Visuals are static and contained in one DZC • Examples: • Dog Breeds • Sport Illustrated Covers Collection is stored as static files and a single logical unit

21. Linked Collections • Limited only by storage complexity • Metadata is static and loaded in parts • Visuals are static and contained in multiple DZCs • Examples: • Pro Sports • AMG Movies Collection is stored as static files and multiple logical units

22. Static Architecture Web Server Request as HTTP GET DZI DZI DZI DZI DZI DZI DZI Response DZC CXML as XML and images DZI DZI DZI DZI DZI DZI DZI File Server

23. One More Demo

24. Dynamic Collections • Unbounded in size • Metadata is dynamic and loaded in parts • Visuals are partially dynamic andcontained in dynamic DZCs • Examples: • Wikipedia Collection is dynamically generated at each view

25. Dynamic Architecture Request at query Web Server DB Server as HTTP GET DZI DZI DZI DZI DZI DZI DZI Response DB DZC CXML as XML and images DZI DZI DZI DZI DZI DZI DZI File Server

26. Putting it All Together • Pivot is backward compatible with the Web • Requests are http get’s • Results are plain files returned over http • The schema is application neutral • Compelling scenarios can be realized with nothing more than data (no code) • Adding a proxy layer to dynamically translate data into CXML allows for nearly unlimited use cases

27. Announcing You’re invited to download Pivot!

28. Limited Invites for PDC Attendees • Download cards available after this session. • Each code is good for 10 installs • What you will get: • Client bits • Access to sample collections • White papers describing solutions • A tool for authoring simple collections • In return, we want you to surprise us with some new and novel uses for Pivot.

29. Some Challenges to Entice You • Front end to desktop search • Front end to Sharepoint • Publically available government data • Collections from existing online services • Data sets from biology • Books from the Internet Archive • Data from Microsoft Codename “Dallas” • Surprise us!

30. Explore, Relate, Search, Discover • Search takes us from many things to one • Browsing moves from one thing to another • Exploration exposes serendipitous relations • Today, we treat these as separate activities • Can we do better by making these part of a more unified experience?

31. Why We Think this is Exciting Web Experience Pivot Experience Focus: you look at many things at once Navigation: you move from one to many, many to one Relations: patterns viewable across a collection Experience: akin to swimming through a network of information • Focus: you look at only one thing at a time • Navigation: you move from one page to another • Relations: links embedded within single pages • Experience: akin to reading a flat document

32. Thanks!