Real-Time Networking: Data, Voice, and Video

1. Real-Time Networking:Data, Voice, and Video Dan Schonfeld Multimedia Communications Laboratory ECE Department University of Illinois Chicago, Illinois

2. Multimedia Communications LaboratoryRashid AnsariAshfaq KhokharDan SchonfeldOliver Yu A Vision To The Future!Real-Time High-Quality Global Video Communications over High-Speed Networks

3. Research Areas Video NetworkingVideo TrackingVideo Retrieval

4. Video Networking Collaborators:Rashid Ansari (UIC)Bulent Cavusoglu (UIC)Tom DeFanti (UIC)Jason Leigh (UIC)Emir Mulabegovic (UIC)Oliver Yu (UIC)

5. Homeland Security Application Testbed • Chicago Video Surveillance • Monitoring of Biochemical Sensor Arrays • Helmet Mounted Cameras and Portable Sensors for Search/Rescue Operations • Dedicated Emergency Telephony Network • Public network lack of reliability during emergency

6. Central Monitoring • Ok as long as links are dedicated • Search and rescue Central Monitoring • Remote sharing with agencies in the field • Mobile and Portable Emergency Response Center Real-Time Surveillance/Monitoring

7. Best Effort Guaranteed & Controlled-Load Non-Real-time Real-time Traffic Control User Application Topology Distribution IP QoS Control Real-time Multimedia Path Selection MPLS Forwarding HTTP/SMTP/FTP/TELNET IP Routing Middleware Signaling Protocol Network & Layer Management BGP CR-LDP IGP RSVP ICMP Current Internet Transport Signaling Transport TCP Out-of-Band UDP Associated NGI Extension RTP ATM/PPP/HDLC IP / MPLS SONET Fiber/DWDM Next Generation Internet Architecture

8. VIPER: FEC/UDP Over STARTAP (Chicago-Amsterdam)

9. X MBZ E f(0,0) T TR f(0,1) AN f(1,0) N f(1,1) S DC B PS E T P P C FBV Q V BFC A R FFV H FFC G D Picture Type MPEG-1 Header extension f_code MPEG-2 Header extension Real-Time Transport Protocol (RTP)

10. AdaptiveFEC

11. FEC Simulations (a) Static FEC (b) Static IPB (c) AFEC (d) Original

12. DiffServ

13. DiffServ Simulations (a) Greedy (b) IPB (c) Motion (d) Original

14. 4 5 7 5 1 4 4 2 6 7 5 2 2 4 2 Rate-Controller

15. Rate-Control Simulations (a) Choke-Only (b) Choke-ORCA

16. Lightweight Streaming Protocol (LSP)

17. LSP PSNR = 18 dBms PSNR = 33 dBms

18. Wireless QoS Control Core QoS Control Internet QoS Control Backbone BS Gateway MSC AMDG-CAC Router Backbone FDWFQ-MAC AP-CAC Backbone Internet CDMA-Based Admission & Scheduling QoS Control Over Wireless & Core Network (Oliver Yu)

19. Listen Idle Duty Cycle 9.30% 8.16% Zzzz Zzzz Zzzz Zzzz 5.13% Zzzz Zzzz Zzzz Call Admission Controller Zzzz 0.56% X Zzzz Zzzz X Channel Slot Level Scheduler Wireless MAC Protocols(Khokhar)

20. Center for Global Multimedia Mobile Communications

21. Internet over Cable

22. Digital Subscriber Lines A typical ADSL equipment configuration.

23. Low-Earth Orbit SatellitesIridium (a) The Iridium satellites form six necklaces around the earth. (b) 1628 moving cells cover the earth.

24. Globalstar (a) Relaying in space. (b) Relaying on the ground.

25. The 802.16 Physical Layer The 802.16 transmission environment.

26. Optical Networking Transport Encoding & Protocols Wired and Wireless Network Integration Circuit and Packet Switched Network Deployment NU UIC NCSA UCSD Computer Network Infrastructure

27. Wireless Communications • Opportunistic Resource Allocation & Admission Control • Channel Estimation • Power-Efficient Wireless Protocols • High-Capacity Wireless Networks

28. Applications & Prototypes • Video Communications • Tele-Education • Natural Event Monitoring • Geosciences Monitoring • Environmental Assessment • Emergency Management • Elderly Care • Medical Diagnosis • Remote Robotic Surgery

29. High-Resolution Scalable Displays High-Resolution Capture Interactive Tools Visualization & Devices

30. Intelligence Sharing Real Time Monitoring and Real Time Multimedia Retrieval and Sharing across the continent

31. Video Tracking Collaborators:Nidhal Bouaynaya (UIC)Karthik Hariharakrishnan (Motorola Research)Dan Lelescu (NTT DoCoMo Research)Josh Meir (NeoMagic)Magdi Mohamed (Motorola Research)Wei Qu (UIC)Philippe Raffy (R2 Technology)Fathy Yassa (NeoMagic)

32. Motivation • Target Tracking • Surveillance • Retrieval • Video Coding • Video Communications • Videoconferencing • Virtual Reality • Human-Computer Interaction • Computer Animation

33. Object cluster Reference frame VORTEX VORTEX: Video Retrieval and Tracking from Compressed Multimedia Databases

34. VORTEX

35. Adaptive Block Matching (ABM)

36. Motion-Based Particle Filters Condensation filter MBPF

37. Multi-Object Particle Filters HMM MHMM

38. Experimental Results

39. IDMOT The Dynamic Graphic Model for Multiple Interactive Objects In Two Frames

40. Punish Reward Magnetic-Inertia Model

41. Video Tracking and Foveation(Ansari & Khokhar)

42. Future Research:Video Tracking • Randomly Perturbed Active Surfaces • Video Stabilization • Auto-Focus Recovery • Pose Estimation and Feature Tracking • Video Animation • Stereography from a Single Camera • Multiple Camera Mosaics • Multiple Camera Tracking • Low-Power Particle Filters

43. Video Retrieval Collaborators:Faisal Bashir (UIC)Ashfaq Khokhar (UIC)Dan Lelescu (NTT DoCoMo Research)Fatih Porikli (Mitsubishi Research)

44. Motivation • Video Surveillance • Sign Language Recognition • Sports Video Analysis • Animal Mobility Experiments • Moving Object Databases • Video and Sensor Databases

45. Spectral Clustering

46. Trajectory Retrieval

47. Gaussian Mixture Models Figure: 1-Sigma contours of GMM’s learnt from three classes. (a) ‘Norway’. (b) ‘Alive’. (c) ‘Crazy’.

48. HMM for Class N HMM for Class 1 Gaussian Mixtures … Classification: … Classification Training Set Database

49. Accuracy Accuracy values for various class sizes from ASL data set and the HJSL dataset (last column). Column headings are shown as (number of classes:number of trajectories) for the ASL dataset at different sizes.

50. Shape Representation