Research Issues in Adapting Computing to Small Devices

1. fcmlau@cs.hku.hk Research Issues in Adapting Computing to Small Devices Francis C.M. Lau Department of Computer Science The University of Hong Kong APPT 2005 • Hong Kong

2. Popular Computing • 70’s: one computer, many users • 80’s: one computer, one user • 90’s: many (connected) computers, many users – distributed computing • Now: many computers, one user • What next? Ubiquitous computing? Grid computing? Service-oriented computing? … • Do we need that many computers? • Do we need a “computer”?

3. Windows Panic • Continue or quit or kill the window or details? • How would my mother know!? • The best computing is without the computer

4. Imagine … • APPT 2015 (Disneyland, Hong Kong) … APPT 2015

5. My Future Handheld • All in one: phone, PDA, wallet, game console, remote terminal, ad-hoc networking … • That means a lot of software – big and fat!? • But what I really want is lean and thin and mean • Lean – simple software which does only what I want • Thin – that’s all the space in my pocket, or around my neck! • Mean – affordable LTM

6. LTM Computing • The (thin-) client-server paradigm, works but in limited ways • Too thin: a browser-only device, a remote terminal … • Future devices are more powerful than just that • But how powerful is powerful?

7. O my god! Software complexity Subramanian, 1999

8. Rich Computing • My hard disk and VM can be as big as the Internet • Not just contents, but software come and go on demand • Pay-per-use subscription model • Browser plug-in’s and Java applets – not lean enough, and some require just too much resources • Software in terms of “functionalities” The Networkis the Computer ™

9. The Network Behind Every Device • Future digicams • Snap and go • The network is the memory • The network is the photo album • (The network is NOT the power) • Context aware

10. Edge Computing • Person .. device .. middleware (proxies) .. Internet • The abstract cloud moves with the client – personalized “cuddleware”, nomadic computing Internet proxies united metropolis client

11. GRID EDGE DEVICE The continuum: WHAT SHOULD BE DONE WHERE?

12. A Continuum • Your device .. other devices (peers) in the vicinity .. server(s) nearby .. the grid • “Neighborhood” is still important • network bandwidth can’t be infinite; latency will never be 0; network will never be trouble-free • What should be done where? • The end-device: completely passive (HAS) to completely active (DIY)

13. “Hire a Server” • Completely passive (remote terminal) • A waste of bandwidth • Responsiveness and interactivity (esp. for games) • Does it make sense? • MP3, camera, PDA, … • Something must be done locally • Adaptation

14. Limits You Can’t Beat • Power? • Screen size: foldable, rollable, wearable, projectable? Or do without the screen (audible?) • Convenience and privacy; and power consumption • Resolution helps a little bit • Why I don’t use a PDA (I just ordered an 24” LCD; but I still look at my watch) • Ergonomics is very important • Scrolling is a nuisance (now) • User experience!

15. Wearable Computers server display

16. Adaptation • Data • = not code • Content • A special kind of data • Text, paged, non-text • Pixel level and/or higher • UI • Higher level, structural, interactive • Buttons, icons, menus, panels and panes … •  content • Code

17. Componentized Software • Executable code as puzzle pieces • Fast start, like contents • Software download pipelining – first-come-first-execute, the most needed functionalities first

18. The Need for Code On Demand • How many functions have you ever used? A simplified view of Microsoft Word

19. Remember UNIX? • Small is beautiful • Make each program do one thing well • Choose portability over efficiency • Store data in flat ASCII files • Avoid captive user interfaces • Make every program a filter • …

20. UNIX Philosophy Applied • Small software components implementing single functionalities • Bigger functions runtime-composable from small components • Download on demand, and disposable • Standard formats and interfaces (eg. XML) • Shared GUI

21. Service-Oriented Computing Can Web Services Scale Down?- for small device apps

22. Adapting X to Y • Many issues • Psychology • Aesthetics • Readability, convenience, privacy … • Economics • Context awareness • Because small devices are likely mobile • The user moves around

23. Awareness and Adaptation • An intelligent supporting architecture • Aware of the come and go of mobile users • Choices of functions, depending on situation: the location, the device, the user, … • Fast reacting nearby proxy • Proxies united, within certain locality

24. Summary 1: Basic Problems (I think) Worth Pursuing • Edge computing → “seamless” • New protocols for the edge • The continuum → the network is the computer • Collaborative models and mechanisms, esp. at the edge • The global grid → invisible, “PC” disappearing • The device • Adaptation • On-demand code composition • The SOC approach? • Content • HTML • UI description languages • New paradigms for user interaction in small devices (input and output)

25. Some HKU Projects • The Sparkle Project – “code adaptation” • PDF document content adaptation system – “content adaptation” • HTML adaptation – “UI/content adaptation”

26. Application = “facet” composition Sparkle • Mobile code on demand • Proxy to device, device to device • Features • Code On Demand • Context Based Adaptation • Proxy Caching on Code • Code & State Migration • Ontology • Sparkle View & Universal Browser

27. 5000 4500 4000 3500 3000 Time (ms) 2500 2000 1500 1000 500 0 Add Noise Add Find Flip Flip Gaussian Open Save as Sharpen Shadow Edges Horizontal Vertical Blur Image Jpeg Universal Browser

28. PDF Document Adaptation

32. Web Document Adaptation

33. The original content • Only the bottom level is “compressed” • (c) The two lowest levels are “compressed”

34. Handling Segments

35. Summary 2: When I Grow Older • I want a simpler life (others can enjoy all the complexities) • One device, LTM • No computer please; terminal is fine; plug and play • Single data storage by hire, access anywhere • Single sign-on, single environment (adaptive)