Dr.-Ing. H. W. Gierlich Head of Telecom Division

1. CITEL (PCC.I)/ ITU Forum on Information and Communication Technology Service: Quality, Control and Surveillance (Cartagena de Indias, Colombia, 23-24 September 2013) QoS assessment for mobile speech services including user interfaces in vehicles and hands-free communication Dr.-Ing. H. W. Gierlich Head of Telecom Division

2. Outline • Introduction • The impact of terminal design on speech quality • Assessment methods and standards for mobile terminals • Driver distraction and speech services in cars • Summary

3. Users Expectation (QoE) Listening Effort Sound Quality & Naturalness Intelligibility Speech Individual Characteristics Listening Talking Effort Conversational Effort Environmental Network Conditions Conditions Double Talk Performance Expectation ... Transmission of Background Noise ... Speech Quality

4. Signal Processing in Modern Terminals NLP - H(f ) + Decoder speech recognition/ control Noise cancellation AGC/ companding … EC Coder EQ Mikrofon –(Array) Modulation / Demodulation NR Coder SE Important: All signalprocessingis non-linear, time variant and targetedtospeechenhamcement Decoder Loudspeaker z.B. GSM/UMTS EC CN Speech- “enhancement” Comfort Noise

5. Speech Quality USERS PERCEPTION …is subjective and generally different between subjects …is based on experience …is cultural dependant TESTING …in general should be based on users perception …may be based on simple signals and procedures modelling aspects of users perception (e.g. loudness) …may be based on complex signals and advanced procedures modelling aspects of users perception (e. g. switching and double talk tests) …may be based on perceptual models using psychoacoustically motivated procedures (models of the human hearing)

6. Outline • Introduction • The impact of terminal design on speech quality • Assessment methods and standards for mobile terminals • Driver distraction and speech services in cars • Summary

7. Speech Quality Evaluation Sending Receiving Echo Performance Quality of Background Noise Transm. Double Talk Speech- Quality

8. Terminal Testing • Bring the world to the lab: • Simulate user • Simulate user behaviour • Simulate user environments • Simulate network and access points • Simulate network conditions • Valdidate correct NTP access

9. The Typical Lab Setup Network Simulation Testsystem Sub-woofer Background Noise Simulation

10. Relevant Standards • GPP TS 51.010 (GSM 11.10) • 3GPP TS 26.131 & TS 26.132 • 3GPP2 C.S0056 • TIA 928 (CDMA/TDMA) • CTIA • GSMA HDvoice standard • For more advanced testing: • ETSI TS 103 737 – TS 103740 standard series

11. Moste Important Test Parameters Enabling a Good QoS Performance Basic acoustic tests e.g. Loudness Ratings (ITU-T P.79), Frequency response tests… Delay and echo tests Speech quality tests – with and without background noise Conversational test including double talk, switching… Important: Test signals are mostly speech based – ITU-T P.501

12. The Impcat of Delay on Conversational Dynamics 0 ms Network Terminal B Terminal A 150 ms 500 ms • Best case between fixed & mobile network (no satelite): • ~150 ms • Worst case across networks: • >> 500 ms For superior conversational quality: delay < ~150 ms (from the users perspective)

13. Conversational/Double Talk Test Signal Female Male Female Male Receive Send Short isolated words Double talk sentences Word in a speech pause Word in partially masked Word fully masked, one speaker continuosly talking Word conicident with the other channel speaker, Simulating delay and unwanted double talk

14. Summary of Quality Parameters Using „Quality Pies“ according to ITU-T P.505 Conv. Quality Conv. Quality Conv. Quality Conv. Quality Background Noise Performance Phone 2 Phone 3 Phone 4 Phone 1

15. Outline • Introduction • The impact of terminal design on speech quality • Assessment methods and standards for mobile terminals • Driver distraction and speech services in cars • Summary

16. Driver Distraction • Physical/manual distraction • Manual operation of devices • Visual distraction • Watching information other than road • Auditory distraction • Focusing on auditory events not related to the driving task • Cognitive distraction • Occupied by non driving related tasks

17. Technology Based Distraction in Cars Car entertainment systems Navigation systems Phones for speech communication Car information systems Text messanging systems „Texting while driving“ :

18. To be Avoided • In general all activities distracting from the driving task, esp.: • Visual cues which require different focus than road • Non intuitive manual operation of the car • Loading the human auditory system by distracting cues • Unnatural and low quality dialogs and communications

19. Driving Task mostly occupied: visual system mainly involved: hands and legs involved: auditory system not involved: talking The driving task => Auditory Channel and Speech “available”

20. Speech Based Services in Cars • The main speech applications: • Speech recognition systems • Speech dialog systems • Text to speech systems • Speech enhancement for communication systems • Hands-free communication • Enhanced in-car communication systems between passengers

21. Communicational Quality – Drivers Distraction Superior Communication Quality -> Reduced Drivers Distraction Perfekt Intelligibility, High Speech Naturalness in the Car Perfekt Intelligibility, High Speech Naturalnessat the Far End Seamless Interaction -> Low Delay, Perfekt Double Talk Performance

22. Hands-Free in Cars • Why hands-free in cars? • Reduce phsyical distraction (dialing, holding the phone…) • Reduce visual distraction (watching display, keyboard …)

23. Superior Sound Quality & Intelligibility in the Car • Wideband services in mobile networks available soon • -> • Enabling wideband telephony (100 Hz- 8 kHz) in cars • Fullband • Narrow band (car) • Wideband (car) • Efficient use of the high quality audio systems in cars: • Getting superior sound quality • Increasing speech intelligibility • Increasing naturalness of a conversation • Reduce drivers distraction due to poor speech sound quality

24. Radio Link Hands-free signal processing Car audio system Phone adapter & antenna amplifier Bluetooth Link

25. Car Hands-Free Testing Mobiles Sound Playback System (can be installed in any vehicle) Test System analysis analysis

26. ITU-T Standards Contributing to Reduced Drivers Distraction • ITU-T standards for Hands-Free Communication: • ITU-T P.1100 for narrowband hands-free • ITU-T P.1110 for wideband hands-free • ITU-T standards work in FG CarCOM: • New work on subsystem requirements for Hands-Free Systems in Cars • ITU-T standards for Dialog Systems: • P.851: Subj. evaluation of dialog systems • Suppl. 24 to P. Rec.: Parameters describing the interaction with spoken dialog systems

27. The Main Section in ITU-T P.1100 & P.1110 Definition of the test arrangement in a car Description of a digital interface concept for development and debugging Microphone test specification for separate microphones in a car Measurement parameters and requirements for hands-free terminals Bluetooth test interface for validation of telephone performance Subjective test strategy for car to car communication

28. Conclusions and Recommendations Mobile terminal testing is essential to provide sufficient or even superior QoE to the user – many operator have specific terminal standards for their suppliers A variety of parameters contribute to the overall (conversational) speech quality Network conditions have to be considered in the testing Low driver distraction is essential to offer mobile services Speech based operation is the key to the success of mobile services in cars Both, terminals and networks have to be thoroughly engineered and adapted to each other

29. Tandeming of Signal Processing in Terminals & Networks Mobile Phone gain EC ES NR Mobile Network Base Station gain EC Netw. 1 ES NR Netw. 2