The Sprint Essistant: Interactive Services Through a Natural UI

1. The Sprint Essistant:Interactive Services Through a Natural UI James Schumacher jim@sprintlabs.com January 2002 Sprint Advanced Technology Laboratories Burlingame, California

2. The Sprint Essistant A federated system of loosely coupled services made cohesive through an innovative multi-modal interface. Leveraging such components as animated agents and Text to Speech engines, the Essistant enables the user to make a personalized connection to the services.

3. What is the Essistant? • Not the desktop • Ubiquitous computing • Smart House / Smart Space • Pervasive Computing • Speech & Transparent User Interfaces

4. LAN as Client • Federated Services • Backend Services feed the LAN • LAN as intermediary • LAN Services function autonomously

5. The LAN & Connectivity • “Broadband is five years away.” • Long live narrowband (DSL & Cable) • Down rates average ~400kb/s • Leveraging “Always On” • Fat Content Over Skinny Pipes • Keep Data Flowing. • Maximize the use of narrowband.

6. Fat Content over Skinny Pipes • Assumptions: • Typical broadband connectivity is 0.3 -1.5 Mbps. • BW bottlenecked pipes not necessarily at the last mile. • Multimedia content (mostly video) is emerging. • High percentage of multimedia content is stored. • Most of multimedia content is currently encoded/delivered at a constant bitrate. • Broadcast quality content > 1.5Mbps • Can this content be sent over a slow pipe?

7. Fat Content over Skinny Pipes • LAN utilization of fat content: • Active Prefetched media • Video-on-demand • Purchase of multi-media (Music, DVDs, etc) • Rich media web sites pre-cached.

8. Services of the Essistant • Fat Content download manager • Various content agents (news, traffic, etc) • Room control (lights, thermostat, security) • Entertainment system (set-top box, VCR) • Chatter-bots (e.g ALICE) • Robots

9. Essistant User Interface • Speech Recognition • Animated Characters • Text-to-Speech response • Face recognition / Motion Detectors • Other sensors (e.g thermostat, microphone)

10. Essistant Sample

11. Enabling Technologies • Connectivity & Bandwidth • Processing Power • Speech Recognition • Text-To-Speech • Real-Time Animation • Face Recognition • Robots • Artificial Intelligence

12. Enabling Technology: Bandwidth • “Broadband is five years away” • What to do with 100mb? • Not just faster web pages • Entertainment, Cable TV, VOD. • Telephony and VoIP

13. Enabling Tech: Mobile Bandwidth • Network is always with you • Mobile, G3 connectivity • Connectivity to home LAN from anywhere

14. Generic Architecture Agent Preference database Intelligent Agent Network Provider User-Agent Net-Agent AI Databases User-Network Home User Prefs User-Agent User-Network Mobile User

15. Enabling Tech: Artificial Intelligence • Speech Recognition • Natural Language Processing • Chatter-Bots (ALICE) • Profiling, and auto preferences • Semantic Web & Ontologies • Inference Engines (e.g Cycorp)

16. Use of AI to predict user requirements • Wide availability of AI engines. • AI capabilities have increased, cost has decreased significantly. • Ideally suited to alter the form of interfaces to the network based on user requirements and user input. • Advanced form of user profiling and personalization. • AI predictions may alter the network usage and its patterns considerably.

17. Current Trends: Personalized Interactive Services • Goal: • Development of a distributed network and client-aware interactive system which integrates and delivers personalized content and services to users in a seamless manner through agent technology and network-based user information. • Adds value by: • Personalizing content. • Personalizing interaction. • Seamlessly introducing new services. • Allowing accessibility by different devices/networks (PCS, broadband, etc.). • Providing for targeted advertisement.

18. Enabling Technology: Animation & 3D • Three dimensional displays • Three dimensional cameras • Dynamic real-time animation

19. Live 3D capture technologies • Z-Cam

20. 3D Display Technology • Stereoscopic monitors • Multi-planar monitors

21. Robotics and Physical Presence • Capabilities of robots have increased at a much lower cost • Virtual Presence • Virtual Telecommuter • Human/Robot remote collaboration (POP troubleshooting…) • Users can have a physical presence 

22. Robotics and Physical Presence (cont’d)

23. MPEG-4 Applicability • Emerging international standard for delivery of multimedia traffic. • Provides: • Delivery of all known media types over any transport mechanism (IP, ATM, RTP, MPEG-2 Transport Stream, etc.). • Scene coherency and synchronization. • Scalability due to bandwidth (BW) or client device limitations. • Interactivity with 3D objects and object-based compression and delivery. • Encode-once/use-everywhere content.

24. High-Level Architecture Agent Client MPEG4/Anim Server Side Client Side MPEG-4 Player MPEG-4 Server component w profiler Animation Engine (Server) w profiler Voice/Face Recognition Signalling path Data path Sprint Controller Profile Manager AAA Billing AI engine Service Control Interaction Engine Content Databases Backend Text-to-Speech Server Service 1 Service 2

25. Current Status • Prototype of a network to user interface. • Platform for test of intent interpretation technologies. • Open user-end platform for quick addition of novel services. • Ongoing Integration with MPEG-4 infrastructure.

26. A Glimpse of a Possible Future… VoD

27. Economic Models • Flat-Rate Pricing currently in use • Usage-Based Pricing? • Service-Based Pricing? • On-Demand Self-Enrollment for Services • Models for Revenue Sharing in “wall-garden” networks with multiple service providers

28. Conclusions • Several technological advances enable the next wave of services based on: • New UIs • Network Intelligence • Advanced personalization • User anticipation • Advanced Services not very far out • Business Models not clear yet

29. That’s all folks! James Schumacher jim@sprintlabs.com