1 / 21

Improving Speech Application Performance: Key Indicators and Case Study Insights

This document outlines the essential indicators for measuring and improving the performance of speech applications, focusing on metrics such as Concluded Calls (COS) and Average Duration (DUR). It presents a case study on National Express in Spain, illustrating how to analyze conversational efficiency through real-world data. By evaluating user interactions and feedback mechanisms, the study emphasizes the importance of perceived quality and operational evolution. The goal is to provide actionable insights for managers to enhance IVR systems and improve user experience through data-driven decisions.

berne
Télécharger la présentation

Improving Speech Application Performance: Key Indicators and Case Study Insights

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring & Improving Performance of Speech ApplicationsBy Pablo Garín Presidentpgarin@natvox.es34-945-227200

  2. Summary • INDICATORS • COS - Concluded calls. • DUR – User comfort. • Comparative Agent Duration versus IVR. • Split Conversation in single operations. • Q • Equivalent job done by the IVR – week. • CASE STUDY • A typical conversation. • Actual IVR in a peak week. • Manually tuned statistics. • IVR evolution. From agents (2002) to fully tuned IVR (2006).

  3. Introduction Indicators Q Case Study • Measuring IVR performance = measuring perceived quality. • Not the same, IVR number of calls versus Agents number of calls. • Perceived quality by interviews is not valid for feedback actions. • Conversational quality indicators have to: • Measure perceived quality. • Give direct feedback to IVR actions. • Be obtained automatically to be consistent in time. • Be manually followed to have validity. • Assist managers to evolve the operations.

  4. COS Indicators Q Case Study • % of concluded calls, initially, it is a good measure. • Considering only incoming calls that want related targets.

  5. DUR Indicators Q Case Study • Average Duration of calls measures user comfort. • Shorter calls mean: • Better conversational strategy. • Better recognition; less iterations. • Less audio to explain things. • In practice, users evolution is slow. • Changes in applications are reflected immediately. • Following DUR evolution is capital. • Average duration of concluded calls.

  6. COS & DUR Indicators Q Case Study • Splitting conversation in pieces allows COS & DUR to take value.

  7. COS& DUR Indicators Q Case Study

  8. COS& DUR Indicators Q Case Study • Each block is named “operations” so COS & DUR are related to operations.

  9. COS & DUR Indicators Q Case Study • For each operation, i.e. schedule information, we measure: • Number of users entering the operation: INI = 28.774 / week. • % of users that conclude the operation: COS = 80,10% • Average duration of operation for concluded calls: DUR = 46,2 secs.

  10. COS & DUR Comparing IVR DUR, for a particular operation, with AGENT-DUR gives CDUR: IVR Dur “Info” = 46,2 secs. Man Dur “Info” = 60,0 secs. CDUR = 130% Indicators Q Case Study

  11. Equivalent IVR Job = = 384 hours / week Q = Indicators Q Case Study For each operation we have (i.e. Schedule): • Number of users entries: 28.744 (week) • COS 80,1% • DUR 46,2 secs. • ADUR 60,0 secs. • CDUR 130%

  12. A typical conversation Indicators Q Case Study Welcome to National Express, good morning. How can I help you? I’d like to know the departures’ time from Valencia to Barcelona Sants. From Valencia to Barcelona Sants you have daily, except bank holidays: A service at 12:00 Another at 18:00 And another one at 20:00 Anything else? Yes, I want to buy the one at six o’clock in the afternoon. For what day? For next Friday. All right, form Valencia to…

  13. Case Study Indicators Q Case Study Next table is statistics for: • National Express, Spain • Call Center in Madrid serving all Spanish calls • Week 7-13 August 2006 - peak

  14. Conversational Efficiency Indicators Q Case Study

  15. Conversational Efficiency Indicators Q Case Study

  16. CASE STUDY Indicators Q Case Study Manual validating indicators. • Real % of concluded operations. • Schedule information, cancel tickets… Real COS is higher. • Reasons for non concluded calls: • Due to IVR. • Due to other reasons.

  17. Conversational Efficiency Indicators Q Case Study

  18. Conversational Efficiency Indicators Q Case Study

  19. Evolution of IVR Indicators Q Case Study Comparison Agents First week August

  20. CONCLUSIONS Indicators Q Case Study • Absolute COS & DUR values measures actual performance. • COS & DUR evolution gives results of application changes. • COS & DUR evolution gives change in operating environment. • Q of each operation gives real relative importance. • IVR Q gives total job performed.

  21. Thanks By Pablo Garín Presidentpgarin@natvox.es34-945-227200

More Related