1 / 55

Doctoral Thesis Proposal

Doctoral Thesis Proposal Viral Entertainment as a Vehicle for Disseminating Core Development Services. Agha Ali Raza Thesis Committee: Roni Rosenfeld, LTI, CMU (Chair) Jaime Carbonell , LTI, CMU Bhiksha Raj, LTI, CMU Umar Saif , ITU, Pakistan Bill Thies , MSRI, India.

tambre
Télécharger la présentation

Doctoral Thesis Proposal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Doctoral Thesis Proposal Viral Entertainment as a Vehicle for Disseminating Core Development Services Agha Ali Raza Thesis Committee: RoniRosenfeld, LTI, CMU (Chair) Jaime Carbonell, LTI, CMU Bhiksha Raj, LTI, CMU Umar Saif, ITU, Pakistan Bill Thies, MSRI, India Language Technologies Institute

  2. Contributors • Babajob.com (Polly-Babajob) • Sean Blagsvedt • Maya Chandrasekaran • Archna Bhatia • MSRI (Polly-MSRI) • Bill Thies • IndraniMedhi • SpandanaGella • IIT Delhi (Polly-JMV) • ZahirKoradia • Aaditeshwar Seth • CMU • Christina Milo • JehanzebSherwani • Guy Alster • Yibin Lin • Haohan Wang • Rita Singh • Christos Faloutsos • DanaiKoutra • Jay Yoon • LUMS/ITU (Polly-Lahore) • Umar Saif • MansoorPervaiz • SamiaRazaq • FarhanUlHaq • Zain Tariq Intro Viral Ent. Conduit Test-bed Reproducible

  3. ICT4D • Information and Communication Technologies for Development • More and better information and communication furthers the development of a society. • Can lead to better management of available resources, improved monitoring & reporting of corruption and more connectivity among people • To achieve impact at a massive scale: • Robust solutions to reach the masses using available means with minimum resource expenditures Intro Viral Ent. Conduit Test-bed Reproducible

  4. How do you reach low-literate masses? • TV, radio are non-interactive • PCs are not feasible • Smart phones are not always feasible • Text is problematic for the non-literate • Speech over simple phones is a viable way to reach • low-literate masses Intro Viral Ent. Conduit Test-bed Reproducible

  5. Long Term Vision • Speech-based information access • for low-literate people • in developing countries • Examples of information services: • speech-based Craig's List • speech-based citizen journalism • speech-based message boards/blogs • health, agriculture, jobs, education,… • … Intro Viral Ent. Conduit Test-bed Reproducible

  6. Problem 1: User Training & Motivation • Speech Interfacesusually require user training • e.g. Healthline(Sherwani et al 2009), AvaajOtalo (Patel et al 2010) • Explicit training is not a scalable solution • Solution: Incentivize people to train themselves • “Where there's a will there's a way…” [Smyth et al 2010] • “Entertainment turns UI Barriers into mere speed bumps” Intro Viral Ent. Conduit Test-bed Reproducible

  7. Problem 2: Mass Dissemination How to advertise / communicate to masses who don’t read? Solution: Viral Spread Intro Viral Ent. Conduit Test-bed Reproducible

  8. Goals • Develop Viral Entertainmentas a vehicle for disseminating Development relatedtelephone based services • Research Objectives • Find the right kind of Entertainment • Use it to Introduce and Popularize speech interfaces • Use entertainment as a Viral Conduit for delivering Core Development Services • Setup an Experimental Test-bed for testing speech interface choices Intro Viral Ent. Conduit Test-bed Reproducible

  9. Research Questions • Can we find a: • simple-to-understand, • non-controversial, • quick-to-engage-and-spread form of entertainment that is suitable for low-literate telephone users formerly inexperienced with automated dialog systems? • Could it become Viral? Intro Viral Ent. Conduit Test-bed Reproducible

  10. Our First Attempt: Songline • A telephone-based, voice-based application which allowed users to listen to songs recorded by others, as well as to record their own songs and to forward them to friends • User Feedback (Pakistan) • About the idea: • Songs and music are considered controversial • Privacy concerns • About the Interface: • Entering phone numbers is tedious • Busy tone is confusing • Detailed call tree and numerous options are confusing Intro Viral Ent. Conduit Test-bed Reproducible

  11. Polly • Polly is a telephone-based, voice-based application which allows users to make a short recording of their voice, modify it and send the modified version to friends. • Features • Non-controversial entertainment • Peer-to-peer • Simple & Easy-to-understand (few options) Intro Viral Ent. Conduit Test-bed Reproducible

  12. 2011-Pilot Launch • Seeded with the 32 users on March 31, 2011 • Remained up for 22 days • Amassed 2,032 users who took part in 10,629 interactions Intro Viral Ent. Conduit Test-bed Reproducible

  13. Research Questions • Can we use Entertainmentas a delivery vehicle for core development services? Intro Viral Ent. Conduit Test-bed Reproducible

  14. 2012 – Large Scale Deployment • Deployed locally in Lahore • Call charges went down to $0.023/minute • Increased capacity to 30 phone lines • Increased number of voice manipulations to six • Added Job Audio-Browser Intro Viral Ent. Conduit Test-bed Reproducible

  15. First Information Service: Job Audio-Browser • Scan Pakistani newspapers for jobs for low-skilled workers • Record these ads • Invite Polly’s users to audio-browse them Ref: paperpk.com Intro Viral Ent. Conduit Test-bed Reproducible

  16. Intro Viral Ent. Conduit Test-bed Reproducible

  17. 2012 – Large Scale Deployment • Launched on May 09, 2012: • Cold seeded with 5 most-frequent users of the pilot • After 141 days (mid-September 2012): • 495,000 calls • 85,000 users • Spreading to 1,000 new people daily • 27,000people used the job search service • listened 279,000 times to job ads • and forwarded them 22,000 times to their friends. Intro Viral Ent. Conduit Test-bed Reproducible

  18. 2012 – Large Scale Deployment Intro Viral Ent. Conduit Test-bed Reproducible

  19. 2012 – Large Scale Deployment Intro Viral Ent. Conduit Test-bed Reproducible

  20. After a Year… • Polly had accumulated: • 636,000 calls • 165,000 users • 34,000people used the job search service • listened 386,199 times to 728 job ads • and 19,000 users forwarded them 34,000times to their friends. Intro Viral Ent. Conduit Test-bed Reproducible

  21. Research Question • Can we use the virally spreading entertainment service as an Experimental test-bed for performing randomized controlled trials and demographic studies? Intro Viral Ent. Conduit Test-bed Reproducible

  22. User Retention • Enthusiasm is Lost Quickly • A few people continue long term Intro Viral Ent. Conduit Test-bed Reproducible

  23. Randomized Controlled Trials • So far we had been paying all airtime charges using “missed call” mechanism • Goal: Reduce our airtime charges while maintaining system spread among the poor • Question:How will various quotas affect user behavior? • Introduced a caller-paid line • Calls picked up, caller pays airtime • Impose various quotas on the number of toll-free calls per day (for each user). • When quota exceeded, direct caller to caller-paid line Intro Viral Ent. Conduit Test-bed Reproducible

  24. Caller-paid Line Significantly less activity as compared to the toll-free line Intro Viral Ent. Conduit Test-bed Reproducible

  25. RCT: Effect of Daily Quota of 7 Calls No appreciable difference after a week Intro Viral Ent. Conduit Test-bed Reproducible

  26. User Demographics • Determined by listening to a sample of recordings: • Used mostly by Punjabi speaking men… Intro Viral Ent. Conduit Test-bed Reproducible

  27. User DemographicsEstimated from 207 survey calls Socio-Economic Status Low SES (No Education) Primarily used by low-educated, low-income people Intro Viral Ent. Conduit Test-bed Reproducible

  28. Geographical Spread • - Seeded in Lahore and Okara • - Reached all parts of Pakistan. • - And also a handful of calls from: • India • Belgium • Oman • Saudi Arabia • UAE Intro Viral Ent. Conduit Test-bed Reproducible

  29. Analysis of User Behavior • Does more experience using Polly lead to: • an improvement in users’ interaction skills? • Use of more advanced features, • Fewer mistakes. • 2. any change in usage preferences? • (Analyses are based on: 50,414 users, 292,951 calls, 934,742 menu interactions) Intro Viral Ent. Conduit Test-bed Reproducible

  30. Barge-in Behavior in Main Menu Clear increase in the use of barge-in with experience Number of times user previously encountered main menu Intro Viral Ent. Conduit Test-bed Reproducible

  31. Research Question • Reproducibility: Can our setup and results be reproduced in a different country? Intro Viral Ent. Conduit Test-bed Reproducible

  32. The BabajobCollaboration(Sean Blagsvedt, Maya Chandrasekaran) • Babajob.com is one of India’s largest informal and entry level job portal that connects registered job seekers and employers using their website (primarily), voice services as well as SMS • (Another benefit: A limitation of our previous study was our inability to track employment outcomes) • Goal: Advertise Babajob to low-literate job seekers through Polly and connect them to Babajobs’ voice services Intro Viral Ent. Conduit Test-bed Reproducible

  33. Setup • A local phone number in Bangalore to receive the missed calls • Polly calls back from the US • Support for Kannada (Archna) and Hindi (Rita) voice prompts • Call transfer to Babajob IVR system from Polly’s main menu. • We added a feature: directory of previously-called friends Intro Viral Ent. Conduit Test-bed Reproducible

  34. Seeding Attempts (via Babajob) • Automated calls to 84 active Babajob users in the Driver category followed by an SMS • Staff members at Babajob volunteered to pass Polly’s number on to 5 of their friends • Results • Very little activity that declined within a few weeks • Very few users; mostly listening to job ads (not much playing or forwarding) • We suspect these users to be JobData brokers Intro Viral Ent. Conduit Test-bed Reproducible

  35. Seeding Attempts (via MSRI)(Bill Thies, IndraniMedhi, SpandanaGella) • On Jan 9, 2014 a set of undergraduate students were asked to spread word about Polly. They posted on their university Facebook pages and message boards • In the last week of Jan, 2014 Spandana gave Polly’s demo to a security guard in her building and asked him to spread the word • Results • Over the last 106 days: • 5,051successful calls • 2,793 call-back requests (from 441 users) • 2,258 message delivery calls (to 699 users) Intro Viral Ent. Conduit Test-bed Reproducible

  36. MSRI Seeding: Viral, but Not Exponential Still Ongoing… • Neither exponential growing nor decaying! • Steady stream of new users • Being used primarily by low-literate users in West Bengal and Delhi Intro Viral Ent. Conduit Test-bed Reproducible

  37. Why isn’t Polly Spreading Exponentially in India? Intro Viral Ent. Conduit Test-bed Reproducible

  38. Back-Channel Spread • Observation: Polly is often introduced to new users not via a Polly message • Indication of this: some users place their first call without ever receiving a Polly message (“Word of Mouth” users) • 47.5% of call-initiators in Polly-LHR pilot • 15% of call-initiators in Polly-LHR full deployment • 18% of call-initiators in Polly-BLR • In person introduction? +demonstration? • Explained in a phone conversation? Intro Viral Ent. Conduit Test-bed Reproducible

  39. Why is Spread Non-Exponential? (Hypotheses) • Introduced as a “free voice messaging system” • Different perception perpetuated by “back-channel spread”? • Introduced to college undergraduates • Different behavior in this SES? • Indians have bad past experience with IVR systems? • SPAM, scams • Complex billing mechanisms due to telecom circles (=“area codes”)? • Linguistic complexity? • “Dead-Skunk” hypothesis: something subtle is broken in the user experience • Cultural differences: • Users expressed less interest in voice mods Intro Viral Ent. Conduit Test-bed Reproducible

  40. Remaining Work • JMV-collaboration (ZahirKoradia, Aaditeshwar Seth) • Jharkhand Mobile Vaani is a citizen radio-over-phone platform that makes recorded radio content accessible through phone calls • Goal: Cross-spread: Polly and JMV • Benefits: • JMV operates in the Hindi-Belt • They already have a trusting user-base • More flexibility of seeding • They have a vested interest in this venture • Questions: • Can Polly grow exponentially in India? • Will Polly-JMV cross-spread?

  41. Remaining Work • Launch Polly-JMV (May 2014) • Complete analysis of Polly-BLR data and remaining experiments (May, June 2014) • Analyze data from Polly-JMV (June, July 2014) • Write it up! (July 2014) • Thesis Defence (end of July, 2014) Thank you! Questions?

  42. Hypotheses regarding Non-Virality • Limitations • I am not on-ground • I do not understand the culture/speak the language • Either some inherent cultural/sociological difference that we do not appreciate • Wrong system image (a voice SMS system) due to incorrect seeding? (Back-channel hypothesis). • Peoples' past experience with IVR systems has left a bad taste? In Pakistan it was an untouched population. • Phobia of pressing keys lest thy may lose money? • Fear of calling/picking up calls from another telecom circle? • More entertainment options? More free messaging options? Is the entertainment void situation among low-SES Indians similar to their Pakistani counterparts? • Busy people, less time.

  43. Hypotheses regarding Non-Virality • Some system bug that we have not been able to isolate • Are we getting all the missed call requests from all the regions? • Is the SMS okay. (No!) • Is the caller ID okay? (it was not. Just fixed.) • Are there pauses? Delays? buggy IVR? (None that we are aware of) • Is the language okay? (we believe so)

  44. BLR Lessons Learned • Microphone vs telephone based prompt recordings • Translation of prompts. Formal vs informal. • Outgoing caller-id problem. • Quick detection of bugs. The silent voice prompt. • Problems in seeding. • On ground presence and understanding the local culture, norms, language.

  45. Caveat: Confounding User Types • This analysis lumps together long-term and short-term users. • Hence, it confounds true learning by any one user with differences between the different user types. • Long-term users may be more adept at using IVR systems to start with.

  46. Changes in User Preferences:(now controlling for user type) • We define 3 user sets and explore changes in their usage patterns as a function of their experience(number of prior Polly calls): • Short-term users: 2,701 users who interacted with Polly exactly5 times. • Intermediate-term users: 1,862 users who interacted with Polly exactly 10 or 11 times. • Long-term users: 1,523 users who interacted with Polly 30+ times.

  47. Menu Choices vs. ExperienceShort-term Users (made exactly 5 calls each)

  48. Menu Choices vs. ExperienceIntermediate-term Users (made exactly 10 or 11 calls) Compared to short-term users; activity starts at a higher level and climbs higher!

  49. Menu Choices vs. ExperienceLong-term Users (made 30+ calls) Compared to intermediate-term users; activity starts at a higher level and climbs higher!

  50. Common Trends among all user sets • - The Tendency to: • 1. Press 2 (forward) starts off at a high value and stabilizes • “Back Channel” • System’s popular image as a messaging system • 2. Press 3 (next effect) increases with experience • System Exploration • Hunting for an effect of choice • 3. Press 0 (re-record) and 1 (repeat) decreases with experience • Adapting to 15 seconds recording limit • Playing with the system, alone or with friends • - Invalid button presses are rare

More Related