1 / 26

Mobile Testing: Brought to You by the Letters A & B

#GHC13. Mobile Testing: Brought to You by the Letters A & B. Leslie Nguyen October 3, 2013. 2013. Presenter: Leslie Nguyen. BS/MS of Computer Science from USC iOS developer Cisco Groupon. Guess which generated more signups?. Variation B with image. Variation A with video.

adolph
Télécharger la présentation

Mobile Testing: Brought to You by the Letters A & B

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. #GHC13 Mobile Testing:Brought to You by the Letters A & B Leslie Nguyen October 3, 2013 2013

  2. Presenter: Leslie Nguyen • BS/MS of Computer Science from USC • iOS developer • Cisco • Groupon

  3. Guess which generated more signups? Variation B with image Variation A with video

  4. What does 40.6% mean for 10 million sign ups? • Improvement of 40.6%  2,880,000 more emails • 10% converted to volunteers  additional 288,000 volunteers • Each email tied to an average of $21  additional $60 million in donations

  5. What is A/B Testing?

  6. Show of Hands • Who works on web stuff?

  7. Show of Hands • Who works on web stuff? • Keep your hand up if you have performed A/B testing on the web

  8. Show of Hands • Who works on mobile apps?

  9. Show of Hands • Who works on mobile apps? • Keep your hand up if you have performed A/B testing for these apps

  10. Why Should You Focus on Mobile? These are the 2013 Q2 percentages of revenue from mobile:

  11. Challenges with A/B Testing for Mobile • Long release cycle • Latency is high • What happens when the app is offline?

  12. A/B Testing Made Easy App Server Device ID Bucket Users Configuration Manager A/B Testing Profile Renders UI Analytics Conversion Calculations Stop Test

  13. Problems – First Launch • A/B testing profile downloads asynchronously at launch • What about first launch? • What about A/B testing the first screen?

  14. Solution • Cache the testing profile • Bundle the app with a testing profile

  15. Goal Tracking & Analytics • Client sends analytics of events to the server • Server makes sense of the analytics • Server determines which variation is winning and how long to run the test

  16. What happens if the app is offline?

  17. Solution – Storing Analytics • Client stores results in local phone database • Client uploads the info when the user goes back online

  18. Consistency • It’s important for a user to have a consistent UI experience • Once a user is bucketed, they should remain there for that session • On mobile, the session can last until the end of the test or until the app is updated

  19. What about the slow release cycle?

  20. Solution – Parameterized Tests • Strings or button colors can be wrapped with the configuration manager • This enables values to be configured remotely by server • Elements can be configured at a future time

  21. Strings and Button Colors [ABTest test:@”login” data:^(NSDictionary *data) { button.title = [data objectForKey:@”title”]; button.backgroundColor = [data objectForKey@”color”]; }];

  22. How do you test a UI Layout:Tab Bar vsCarousel?

  23. Solution – Runtime Class Instantiation Class viewClass = [ABTestclassForTest@”Carousel”]; UIViewController *vc = [[viewClassalloc] init];

  24. Conclusions • A/B testing is awesome! • Mobile on the rise • Data driven decisions means better conversions  more revenue • It’s easy to get started

  25. Got Questions? leslie@box.com

  26. Got Feedback? Rate and Review the session using the GHC Mobile App To download visit www.gracehopper.org

More Related