User-perceived QoS in networked multimedia services

1. User-perceived QoS in networked multimedia services Harald Øverby, PhD 28/10-2008

2. Topics • Quality of Service (QoS) • User-preceived QoS • Networked multimedia services • Voice-over-IP (VoIP) (1 hour) • Shared packet redundancy in VoIP • Networked online games (1 hour) • Real Time Strategy (RTS) game Warcraft III How does QoS influence user experience in networked multimedia services?

3. Resources • Y-W Leung, “Shared packet loss recovery for internet telephony”, IEEE communications letters vol. 9, no. 1, january 2005 • P. O. Osland et al., “Perceived VoIP quality under varying traffic conditions”, Proceedings of the Nordic Teletraffic Seminar (NTS), 2004 • N. Sheldon et. al., ”The effect of latency on user performance in Warcraft III”, Proceedings of the 2nd workshop on Network and system support for games • Wikipedia: VoIP: http://en.wikipedia.org/wiki/Voice_over_IP

4. Voice over IP, IP Telephony, Internet Telephony, … Packet switched telephony service Independent packets routed over a packet network Alternative to circuit switched telephony service End to end circuits established for each call ”Stupid” network vs Intelligent network What is Voice over IP (VoIP)

5. VoIP functionality • Voice is encoded in packets, and transmitted when necessary • Ability to transmit more than one telephone call down the same broadband-connected telephone line. • Many VoIP packages include PSTN features that most telcos normally charge extra for, or may be unavailable from your local telco, such as 3-way calling, call forwarding, automatic redial, and caller ID. • VoIP can be secured with existing off-the-shelf protocols such as Secure Real-time Transport Protocol. • VoIP is location independent. • VoIP phones can integrate with other services available over the Internet.

6. QoS in VoIP • QoS definition (one of many…) • Degree of compliance of a service to the agreement that exists between the user and the provider of this service. • QoS (in broad sense) encompasses • Performance • Dependability • Security • In this talk we focus on Performance aspects of VoIP

7. Performance aspects of VoIP • Performance is characterized by a set of paramters, which can be measured quantitatively. • Important performance related QoS parameters in VoIP includes • Available bandwidth • Packet loss • End-to-end delay • Delay jitter

8. User-perceived QoS • How the user perceives the service quality • Measured using e.g. questionaires • E.g.: Will the user see difference between 0 end-to-end delay and 0.1 ms end-to-end delay? • Measured using Mean Opinion score (MOS) • 5: Max, 1: Minimum • Influenced by QoS parameters

9. 1. Available bandwidth • Measured in bps • Depends on the codec • How the voice signals are decoded • Trade-off between complexity, bandwidth savings and other performance related QoS parameters • Bandwidth requirement dependent on the codec • H.323 and SIP are popular protocols • Overhead may constitute a significant part of total bandwidth consumption • E.g. data=5.6 kbps, overhead=12.4 kbps, total=18 kbps

10. 2. Packet loss • Measured in ratio of lost packets to all packets sent • Caused by • Bit errors • Queue overflow in routers • End-to-end delay for packet too high • Acceptable levels of packet loss ratio is ~2 % for VoIP • Dependent on the codec

11. 3. End-to-end delay • Average delay between a user say a word until the received hears the same word • The end-to-end delay is comprised of: • Local computer delay • Compression delay • Packetizing delay • Transmission delay • Propagation delay • Queueing delay • … • End-to-end delay below 150 ms is acceptable

12. 4. Delay jitter • Variations in the end-to-end delay • Difference in the end-to-end delay between independent packets • Caused by • Variations in the queueing delay • Packetizing delay • Computer processing delay • Delay jitter below 50 ms is acceptable for VoIP

13. Shared packet redundancy in VoIP • Novel mechanism to reduce the packet loss ratio (PLR) • Adding redundancy packets to a set of data packets • Requires additional bandwidth • Influences the end-to-end delay and delay jitter? • Complexity issues?

14. Architecture • Each gateway serves N users • Packets arrive to the gateway in slots • n packets in a certain time slot • The gateway has B channels in each time-slot • Packets may be lost in the Internet

15. Architecture slot5 slot4 slot3 slot2 slot1 Gate- Way User1 Data1_5 Data1_3 Data1_1 User2 Data1_3 Data1_2 B=2 User3 Data1_5 Data1_3 Data1_2 n=2 n=0 n=3 n=2 n=1 N=3

16. Shared packet redundancy • A set of n data packets, add r redundancy packets • Transmit all n+r packets • If received at least n packets, reconstruction of lost data packets is possible. • Example: • From 3 data packets we create and add 2 redundancy packets • All 5 packets are transmitted • If we receive at least 3 packets (among 5 packets), reconstruction is possible -> no data packets are lost

17. Example • 2 data packets, 1 redundancy packet • Redundancy packet generated using bitwise XOR in data packets • Red=Data1Data2 • If Data1 is lost, it can be reconstructed as Data1=Data2Red • If Data2 is lost, it can be reconstructed as Data2=Data1Red • If two or more packets are lost, reconstruction is not possible • With more redundancy packets, reed solomon codes must be used

18. Shared packet redundancy in VoIP Router Sent Received Reconstruction data1 data1 data1 data2 data2 data2 data3 data3 data3 data3 red1 red1 red1 red2 red2 red2

19. Parameters • N: number of customers served • n: outgoing voice streams • B: available capacity (slots) • e: Packet loss ratio • : probability that a session is active • The probability that a user has a packet to send • A: activity probability (probability of n active voice streams)

20. n=B: No modifications n>B: Drop packets n<B: Add B-n redundancy packets Erasure coding Packet set N=4 n=3 with prob. An  B=5

21. Erasure conding in VoIP • Always transmit B packets in a packet set • Case 1: n data packets and B-n redundancy packets • Packets are lost if less than n packets are received • Else, lost data packets may be reconstructed • Case 2: n=B data packets and 0 redundancy packets • Lost packets cannot be reconstructed • Packet sets are independent

22. Analysis

23. Analysis: Case 1: n<B • n voice (data) packets are transmitted • B-n redundancy packets are added to the data packet set. • A packet is lost when • It is dropped in the network AND • Less than n of the remaining B-1 packets are dropped

24. Analysis: Case 2: n≥B • No redundancy packets are added • Packets are lost due to • Overflow in the server • In the network with probability e • Example • B=80, n=100, e=0.1 • P(E|An)=(100-80)/100+80*0.1/100=0.28 • Of 100 packets: 20 are lost at server, and 8 in the network

25. Analysis • Overall loss formula • Summing over all n

26. Results B=100 =0.5

27. Qualitative evaluation of the shared packet redundancy scheme for VoIP

28. Topics • Quality of Service (QoS) • User-preceived QoS • Networked multimedia services • Voice-over-IP (VoIP) (1 hour) • Shared packet redundancy in VoIP • Networked online games (1 hour) • Real Time Strategy (RTS) game Warcraft III How does QoS influence user experience in networked multimedia services?

29. Motivation • Computer/video games have become a multi-billion industry and is a driving force for technology development: Hardware, Software, Networks, etc. • Games are played by people of all ages - female and male. • Games are used in serious applications areas: Education, Work training, Physical exercise, Social training, Brain training, etc.

30. What make games intriguing? • Characteristics of enjoyable activities (playing games) can be divided into three categories: • Challenge: • Mastering a challenge can give a self-esteem boost. • Ideally the game should adjust to players abilities. • Fantasy: • Fantasies and abstractions enhance and make the experience more interesting. • Curiosity: • Curiosity is the motivation to learn and investigate. • Desire to bring completeness, consistency to their knowledge

31. Game genres • Action games - shooting games, platform games • Adventure games - logic puzzles in a virtual world • Fighting games - games involving combat • Puzzle games - e.g. Tetris • Role playing game - players assume the role of imaginative creature or person • Simulations - the player is put in control of simplified process. E.g., flight simulators, SimCity, driving simulation etc. • Sport games - games involving sports • Strategy games - typically commanding armies and warfare planning.

32. Online Gaming in General • Most new-released games for PCs and game consoles (PS3, Xbox 360 and Wii) provide online game functionality. • For some games, online-features are main parts of the game (Battlefield) • For other games online-features are add-on functionality (Burnout) • The number of player supported varies from game to game (genre) • For consoles, Xbox Live and Playstation Network (PSN) are driving forces for online game functionality/support. • Xbox Live and PSN have initiated series of simpler online games XBOX Live Arcade, and PSN games (often retro games). • XNA from Microsoft enables independent developers to create own online games distributed on Xbox Live.

33. Main Challenges for Online Gaming in General • Latency (main challenge!) • Ensure smooth gameplay and that other players move realistically (not in steps, but in one motion). • Ensure that variation in latency does not affect players performance/scorein the game. • Bandwidth • Ensure that variation in bandwidth does not affect players performance/score in the game. • Heterogeneity: • Support various types of hardware (PC, consoles) and services (Xbox Live, Playstation Network, etc)

34. Mobile Online Gaming • Mobile online game functionality is becoming more and more common on Playstation Portable and Nintendo DS (add-on to original game): E.g., Killzone: Liberation, Test Drive Unlimited • Few online games for mobile phones: • Online highscore lists • Turn-based and slow-paced games • MMOG on mobile phones in Japan: Samurai Romanesque • Mobile Online Gaming has high potential as there are plenty of users and the users are “always” online.

35. MMOG/MMO • Massively multiplayer online game (MMOG/MMO): Computer game which is capable of supporting hundreds or thousands of players simultaneously. • Games played over the Internet • Feature at least one persistent world. • Enable players to cooperate and compete on a grand scale. • Most MMOGs require a monthly subscription fee. • Huge success last decade

36. Types of MMOGs (Massive Multiplayer Online Games) • MMORPG (Role-playing Game): Guild Wars, FF XI, WoW, EverQuest. • FPS (First-Person Shooter): Counterstrike, Call of Duty. • RTS (Real-Time Strategy): Shattered Galaxy, Boundless Planet, Dreamlords, Ballerium. • MMODG (Dance/Rhythm Games): DANCE! Online, O2 Jam, Audition Online. • MMOR (Racing): Test Drive Unlimited, RaceLands, KartRider. • MMOSG (Social/Strategy/Sports Game): Second Life, Home, Empire. • MMO Real-words simulations: flight-simulations, traffic control, military, etc.

37. Main Challenges for Massive Multiplayer Online Games • Scalability/bandwidth to servers! (Main challenge) • Allow several players to play simultaneously in the same game world. • Divide the world into zones served by separate servers. • Run same game worlds/zones in parallel. • Consistency • Ensure that changes to the game world is consistently distributed among all players. • Latency • Ensure that the players have a responsive gameplay and that game objects move around smoothly. • Bandwidth to clients • Used to be a challenge when players used modem. With broadband connections this is no longer a big challenge.

38. The Effects of Latencyon User Performance in Warcraft III Nathan Sheldon, Eric Gerard, Seth Borg, Mark Claypool, Emmanuel Agu Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA http://www.cs.wpi.edu/~claypool/papers/war3/

39. Network Games and Latency • Latency degrades performance of interactive applications • Web-browsing – seconds • Audioconference – 100’s of milliseconds • First Person Shooters (FPS) – 100’s of milliseconds • Real-Time Strategy (RTS)? • Knowing effects of latency useful for • Building better network games • Building better networks to support games (QoS)  Effects of Latency on Warcraft III (RTS)

40. Warcraft III Overview • RTS User Interaction • Components: • Exploration • Building • Combat

41. Exploration Map • Performance? • Time • (to reach end)

42. Building Map • Performance? • Time • (to build tech- • nology tree)

43. Combat Map • Performance? • Games Won • Unit Scores

44. Controlling Latency • Warcraft III uses client-server • Set computer B as server (also a client) • Set computer C or D as client • NIST Net on computer A • Induce latency [0 ms to 3500 ms]

45. Building and Latency

46. Exploration and Latency

47. Combat and Latency (1)

48. Combat and Latency (2)

49. Bandwidth LAN vs. Internet 3.8 Kbps 4.0 Kbps 6.8 Kbps

50. User-Level Analysis • 0-500 ms latency, users could easily adjust • 800+ ms, game appeared erratic • Degradation in gaming experience • 500-800 ms degradation depended upon • User • More skilled were more sensitive • Strategy • Micro managers were more sensitive • Combat managers were more sensitive