Streaming Performance over Third Generation Cellular Networks

1. Streaming Performance over Third Generation Cellular Networks Author: Musa AldarawshehSenior I&V Engineer/ Nokia NetworksSupervisor: Prof. Sven-Gustav HäggmanInstructor: M.Sc. Karl Tigerstedt

2. Contents Introduction PS streaming QoS theory QoS Key Performance Indicators Environment Description Measurements Tools Preliminary Streaming QoS Tests Streaming Measurements Using Nokia Internal Streaming Server Streaming Measurements Using Helix Universal Server-Mobile Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Streaming in Highly Loaded Cell With Conversational Class Users Streaming Performance Measurement in Mobile Situation GPRS Attach and PDP Context Activation Times Summary & Conclusions References

3. Introduction • This work is aimed to verify the performance of multimedia streaming service over a WCDMA network from end-user perspectives. The measurements were carried out over streaming and interactive QoS class in stationary and mobile situation, the mobility measurements include streaming during soft handover. • The streaming QoS will be verified through a set of Key Performance Indicators (KPI). These includes call setup and call quality KPIs. • Multimedia streaming verification was carried out in Nokia WCDMA Test Network, in Espoo (Finland) during weeks 45 - 52 2004.

4. PS Streaming QoS Theory • Multimedia streaming is a technology that allows compressed live or pre-recorded video and audio content to be delivered to the end user [Lai_p398]. Streaming service considered as real time service because the user plays back the content while downloading. Examples of streaming services are video-on-demand ,web-radio, and news streams. • Media can be divided into two types continuous and discrete media,The first type is only handled in this report To establish a streaming session there are different protocols involved to set-up the connection between the streaming server and the streaming client, They are : • RTSP (Real Time Streaming Protocol) is used to for session set-up and session control • RTP (Real Time Protocol) is used to transport media data (video/audio) over unreliable connection (UDP/IP). • RTCP (Real Time Control Protocol) used to carry information on the participants and session quality. RLC transfer modes for PS Streaming • RLC operates in one of three modes. These are Acknowledge mode (AM), Unacknowledged mode (UM) and transparent mode(TM). • The possible RLC transfer modes for PS Streaming services are either UM-Unacknowledged mode or AM-Acknowledged mode Token Bucket Algorithm: Token bucket algorithm is a reference algorithm to conform the definition of bit rate, see [23107].

5. QoS Key Performance Indicators • QoS is the ability of the network to provide a service with assured service level to the user of the network • QoE (Quality of user Experience) is how an end user perceives the usability of a service when on use (how satisfied he/she is with the service usage) • The following QoS KPIs has been defined to measure the performance of multimedia streaming over WCDMA networks. • Session setup time :This is the time period when the streaming client requests media content from the streaming server until the time when the session setup between client and server is ready and the media start to flow. • Initial Buffering time :This is the time elapsed between the completion of session set-up and the time that media starts playing at the client's device. This time is dependent on client memory storage. • Total set-up time : This is the time when the user request a media content form the streaming server until the time the media is starting to be displayed in the streaming client. • Delay Experienced by the End User: This is the time taken to establish a certain service from the point when the user request the service until the user start to receive information once the service is established. The delay experienced by a streaming service user is the sum of GPRS attach time, PDP context activation and total setup times.

6. QoS Key Performance Indicators • Packet Delay Variation (Jitter): This is the delay variation of arrived packets. • Packet losses: The number of media packets (video/audio) lost during the streaming session. • Throughput: The number of received bits per unit time. Total throughput includes (media + headers) throughputs. • Number and length of rebufferings: This measure is of great importance to the user because the rebuffering operation can stop the audio/video playback. In this thesis only the rebuffering periods that cause a pause in video/audio playback are taken into account. • Subjective Audio/Video quality: The following simple rating scale is defined through streaming QoS measurements • Excellent • Good • Fair • Poor • Bad • Lip synchronisation: This metrics is used to measure the synchronization between audio and video. The metrics can be subjective and objective. The former can be based on the following rating scale. • subjective Lip synchronization • In synch • Just noticeably out-of-synch • Unacceptably out-of-synch • The later can be calculated in the streaming client as the ratio between displayed frames to decoded frames (Lip synchronization index)

7. Environment description • The measurements were done in the NTN (Nokia Test Network), in Leppävaara area during weeks 45-52 , 2004

8. Measurements Tools • Nokia OSS system for logging cell load information • Nemo Outdoor • Global Positioning System (GPS) • Nokia 3G Test Mobile “ Nokia 7600 model” • Nokia Traffic Generator Tool • Helix Mobile Producer • Commview Packet Analyser • Ethereal Packet Analyser • Matlab Tool • Excel • Windows Grep • Nokia Internal Streaming Client • Nokia Internal Streaming Server • Helix Universal Server -Mobile

9. Preliminary Streaming QoS Tests Objectives: Preliminary evaluation of multimedia streaming over Nokia WCDMA Test Network with different client –server streaming solutions. Test Description: Four tests were executed by using two different streaming client–server solutions, two tests for each solution. The client-server streaming solutions are: • Nokia Internal Streaming Client and Nokia Internal Streaming Server • Nokia Internal Streaming Client and Helix Universal Server-Mobile The test was done in stationary situation near to the WCDMA base station where the carrier to interference ratio Ec/No > -6 dB. Two different RT PS RABs were used, 64/64 and 64/128 UL/DL. Multimedia files were used with bit rates 51 and 115 kbps over 64/64 and 64/128 bearers respectively.

10. Preliminary Streaming QoS Tests Preliminary test 1:Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming The test results are presented in Table 1. Average video/audio packets sizes are 450 and 150 byte respectively Table 1 Streaming performance over 64/64 RT RAB The results show a good performance of multimedia streaming when using Nokia Streaming Client –Server. The MOS was “Good” for both audio and video.

11. Preliminary Streaming QoS Tests Preliminary test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming The test results are presented in Table 2. Average video/audio packets sizes are 381 and 313 byte respectively Table 2 Streaming performance over 64/128 RT RAB The results show a satisfactory performance of multimedia streaming when using Nokia Streaming Client –Server solution. The MOS was “Fair” for both audio and video. Video packet losses were in acceptable level.

12. Preliminary Streaming QoS Tests Preliminary test 3: Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Helix Server The test results are presented in Table 3. Average video/audio packets sizes are 485 and 150 byte respectively. The results show a slightly high video packet losses and low audio packet losses, high video packet losses explain the poor perceived video quality Streaming Server sending bit rate more than guaranteed channel bit rate Table 3 Streaming performance over 64/64RT RAB Figure 1 Helix Server throughput over 64/64 RT bearer

13. Preliminary Streaming QoS Tests Preliminary test 4: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Helix Server The test results are presented in Table 4. Average video/audio packets sizes are 333 and 150 byte respectively. The results show a very high video/audio packet loss, which explain the poor perceived video/audio quality. Streaming Server sending bit rate more than guaranteed channel bit rate Table 4 Streaming performance over 64/128RT RAB Figure 2 Helix Server throughput over 64/128 RT bearer

14. Preliminary Streaming QoS Tests • The streaming performance over 64/64 and 64/128 UL/DL real time bearers was bad when using Helix server. • The streaming performance over 64/64 and 64/128 UL/DL real time bearers was satisfactory when using Nokia internal streaming server. The tests had shown that streaming server was very often sending media packets with short delays and the data rate exceeded the channel maximum bit rate. Data rate that exceed the maximum channel rate will be discarded according to token bucket algorithm. The current implementation of token bucket algorithm is done according to 3GPP specification [TS23.107 Annex B], which is not handling appropriately the burst traffic situation. As a result of these findings the token bucket algorithm was modified to be able to tolerate traffic burstiness conditions. This modification allows streaming class to use traffic shaping functionality, which smooth the traffic pattern and decrease the burstiness of the network. All the measurements that follow were done with the modified token bucket algorithm except SHO measurements.

15. Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation Test 1: Multimedia streaming over 64/64 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server AT command used :at+cgeqreq=1,1,64,64,64,64,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 589 and 750 respectively. Average video and packet sizes are 485 and 150 byte respectively. Table 5 Call setup QoS KPIs over 64/64 streaming bearer, BLER 1% The end user will experience the following delays before start to see the movie e.g in trial 1 the end user experience the following delays Attach time+PDP context time + setup time + buffering time = 6.7+6.53+5=18.23 sec See GPRS attach and PDP context measurements.

16. Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Table 6 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 streaming bearer, BLER target 1% Figure 3 shows the total throughput (media + header throughputs) received by the streaming client. The received throughput is within the channel bandwidth in most of the times. Figure 3 Streaming client total throughput over 64/64 RT bearer

17. Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation Test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 2136 and 310 respectively. Average video and packet sizes are 333 and 150 byte respectively. Table 7 Call setup QoS KPIs over 64/128 RT streaming bearer, BLER 1%

18. Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Table 8 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 streaming bearer, BLER target 1% Packet losses are in acceptable level Subjective QoS was good Lip sync index shows a satisfactory values No pause and rebufferings occurred during the streaming session. The received throughput is within the channel bandwidth in most of the times. Figure 4 Streaming client total throughput over 64/128 RT bearer

19. Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation Test 3: Multimedia streaming over 64/256 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average video and packet sizes are 341 and 321 byte respectively. Table 9 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %

20. Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Table 10 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 streaming bearer, BLER target 1% Figure 5 shows the total throughput received by the streaming client. The total throughput was exceeding the channel bandwidth in most of the times . Subjective video/audio quality was not totally satisfactory Lip sync index shows a very low values which indicates that the video and audio are out of sync Many rebufferings occurred during the video. Figure 5 Streaming client total throughput over 64/128 RT bearer

21. Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Objectives: Total number of video and audio packets sent by streaming server is 2136 and 300 To verify the performance of multimedia streaming over NRT bearer in a stationary situation Test 4: Multimedia streaming over 64/64 kbps UL/DL NRT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server respectively. Average video and packet sizes are 333 and 321 byte respectively. Table 11 Call setup QoS KPIs over 64/64 NRT streaming bearer, BLER 1 %

22. Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Table 12 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 NRT bearer, BLER target 1% No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during the streaming session.

23. Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over NRT bearer in a stationary situation Test 5: Multimedia streaming over 64/128 kbps UL/DL NRT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server Total number of video and audio packets sent by streaming server is 2136 and 300 respectively. Average video and packet sizes are 333 and 321 byte respectively. Table 13 Call setup QoS KPIs over 64/128 NRT streaming bearer, BLER 1 %

24. Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Table 14 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 NRT bearer, BLER target 1% No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during the streaming session. The received throughput is exceeding the channel Bandwidth sometimes Figure 6 Streaming client total throughput over 64/128 NRT bearer

25. Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over NRT bearer in a stationary situation Test 6: Multimedia streaming over 64/384 kbps UL/DL NRT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server Total number of video and audio packets sent by streaming server is 4066 and 301 respectively. Average video and packet sizes are 333 and 321 byte respectively. Table 15 Call setup QoS KPIs over 64/384 NRT streaming bearer, BLER 1 %

26. Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Table 16 End-to-End and objective/subjective video/audio QoS KPIs over 64/384 NRT bearer, BLER target 1% Packet losses ~ 1% Subjective audio/video QoS was satisfactory Lip sync index shows a relatively low values The received throughput is exceeding the channel Bandwidth very often Figure 7 Streaming client total throughput over 64/384 NRT bearer

27. Streaming in Highly Loaded Cell With Conversational Class Users Objective: To verify that the guaranteed bit rate of a streaming class user is kept in a highly loaded cell. Test Description: The test was executed by starting a single video streaming session with a predefined data rate and loading the cell with conversational class users (voice calls). The throughput of streaming user was monitored. Environment: Increasing common channels power by 11 dB

28. Streaming in Highly Loaded Cell With Conversational Class UsersTest 7: Streaming over 64/64 RT bearer High BLER DL BLER Throughput decreased significantly due to packet losses Throughput PPP Layer CPICH RSCP UE Tx Pwr CPICH Ec/No Total # of send packets =1389 Total # of packets lost = 15 Packet losses ~1%

29. Streaming in Highly Loaded Cell With Conversational Class UsersTest 7: Streaming over 64/64 RT bearer UL RT Load New AMR calls are not admitted after 33 existing AMR calls PtxTotal DL=~ Ptxtarget DL

30. Streaming in Highly Loaded Cell With Conversational Class UsersTest 8: Streaming over 64/128 RT bearer Throughput is not affected by cell congestion DL BLER Throughput PPP Layer CPICH RSCP UE Tx Pwr CPICH Ec/No Total # of send packets =8118 Total # of packets lost = 36 Packet losses 0.4% New AMR calls are not admitted after 37 existing AMR calls

31. Streaming in Highly Loaded Cell With Conversational Class UsersTest 9: Streaming over 64/256 RT bearer DL BLER Throughput PPP Layer CPICH RSCP UE TX Power CPICH EcNo Total # of send packets =8247 Total # of packets lost = 34 Packet losses 0.4% New AMR calls are not admitted after 37 existing AMR calls

32. Streaming Performance Measurement in Mobile Situation • Objective: To verify the performance of multimedia streaming over real time bearer during soft handover in Nokia WCDMA Test Network. The description of tests is as follow: • Two tests were executed over two streaming class bearers, one test per bearer. The bearer bit rates are 64/64 and 64/128 kbps UL/DL. BLER target value was set to 1% for all tests. Multimedia files were used with bit rates 103 and 213 kbps over 64/128 and 64/256 RT PS RABs respectively. The tests were conducted by using Nokia internal streaming client and Nokia streaming server. Streaming performance was verified during soft handover. The carrier to interference ration was “Ec/No > -6” dB. Every test is repeated 2-3 times to ensure reliability. This test was executed with original token bucket algorithm Softer handover Soft handover

33. Streaming Performance Measurement in Mobile SituationStreaming over 64/128 RT bearer Objectives: To verify the performance of multimedia streaming during SHO Test 10: Multimedia streaming during SHO over 64/128 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming server AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 1880 and 296 respectively. Average video and packet sizes are 382 and 313 byte respectively. Table 17 Call setup QoS KPIs over 64/128 streaming bearer, BLER 1 %

34. Streaming Performance Measurement in Mobile SituationStreaming over 64/128 RT bearer Table 18 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 RT bearer during SHO, BLER target 1% Video packet losses are high due to token packet algorithm problem Lip sync index shows a satisfactory values No rebufferings occurred during the streaming session. Figure 8 Streaming client total throughput over 64/128 RT bearer

35. Streaming Performance Measurement in Mobile SituationStreaming over 64/128 RT bearer Soft handover takes place between different base stations sites within one RNC (intra-RNC soft handover) or within separate controlled RNCs (inter-RNC soft handover). In softer handover the UE is controlled by at least two cells belonging to the same base station site. Soft HO Softer HO

36. Streaming Performance Measurement in Mobile SituationStreaming over 64/256 RT bearer Objectives: To verify the performance of multimedia streaming during SHO Test 11: Multimedia streaming during SHO over 64/256 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming server AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average video and packet sizes are 341 and 321 byte respectively. Table 19 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %

37. Streaming Performance Measurement in Mobile SituationStreaming over 64/256 RT bearer Table 20 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 RT bearer during SHO, BLER target 1% Video packet losses are high due to token packet algorithm problem Subjective audio/video QoS was satisfactory Lip sync index shows satisfactory values No rebufferings occurred during the streaming session. Figure 9 Streaming client total throughput over 64/256 RT bearer

38. Streaming Performance Measurement in Mobile SituationStreaming over 64/256 RT bearer Softer HO Soft HO

39. GPRS Attach and PDP Context Activation Times Objectives: To measure the GPRS attach and PDP context activation times over streaming class bearers. Three different RAB bit rates are used, 64/64, 64/128, and 64/256 kbps UL/DL. BLER target value was 1% for all tests. Every test was repeated 15 times to ensure reliability. Attach Times were calculated as the difference time between ATTACH_REQ and ATTACH_COMPLETE messages. PDP context activation time was calculated as a time difference between PDP_context_create_req and PDP_context_activate_ind messages Table 33 shows that average PDP context + GPRS attach time for 64/64, 64/128, and 64/256 are 6.71, 7.09, and 7.28 seconds. Table 21 Average GPRS attach+ PDP Context Activation.

40. Summary & Conclusions • The preliminary evaluation tests using Nokia internal streaming server had shown a good performance of multimedia streaming over 64/64 UL/DL real time bearer and a satisfactory performance over 64/128 UL/DL real time bearer. • The preliminary evaluation tests using Helix streaming server had shown a bad performance of multimedia streaming over 64/64 and 64/128 UL/DL real time bearers. The following conclusions are drawn from the preliminary evaluation tests: • Nokia streaming server has shown a better performance than Helix streaming server. • The bad performance of multimedia streaming over streaming class was due to the large number of discarded packets by the token bucket algorithm. The current implementation of token bucket algorithm, which is according to 3GPP specification was not suitable to control the traffic flow of streaming servers. • The results obtained after token bucket modification have shown a good performance of multimedia streaming over 64/64 and 64/128 real time bearers and the performance over 64/256 UL/DL RT bearer was not totally satisfactory. • The performance of video streaming over 64/64 and 64/128 NRT bearers was good. Video and audio quality was good and video/audio packet losses were 0.0% in all trials.

41. Summary & Conclusions • The performance of video streaming over 64/384 UL/DL NRT bearer was satisfactory. • The performance of multimedia streaming over 64/128 and 64/256 UL/DL RT during SHO was satisfactory. (original token bucket algorithm + Nokia streaming server) The following conclusions are drawn from the streaming verification tests after token bucket algorithm modifications. • The performance of video streaming over real time bearer was significantly improved after token bucket algorithm modification. • Multimedia streaming over NRT bearers in unloaded cell show a slightly better performance than streaming over RT bearers. • The streaming service user will experience a delay of ~20 seconds before the media contents starts to be displayed in the streaming client. This delay includes attach, context, and total setup times.

42. References [Hol] Harri Holma and Antti Toskala, ”’WCDMA for UMTS Radio Access For Third Generation Mobile Communications”, Second Edition, John Wiley & Sons, LTD, 2002 [Lai] Jaana Laiho, Achim Wacker and Tomas Novosad, “ Radio Network Planning and Optimisation for UMTS”, John Wiley & Sons, LTD, 2002, 484 pages. [22105] 3GGP TS 22.105 Service aspects; Services and Service Capabilities, version 4.1.0, January 2001, 31 pages. [23107] 3GPP Technical Specification 23.107: QoS concept and Architecture, version 4.6.0, December 2002, 38 pages