1 / 33

by Peep Laja, ConversionXL

by Peep Laja, ConversionXL. 15 A/B testing mistakes I see all the damn time (and what to do about it). Call to Action Conference 2014. “You should test that!”. “Dude wtf?”. @peeplaja. Typical scenario:

Télécharger la présentation

by Peep Laja, ConversionXL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.


Presentation Transcript

  1. by Peep Laja, ConversionXL 15 A/B testing mistakes I see all the damn time(and what to do about it) Call to Action Conference 2014

  2. “You should test that!”

  3. “Dude wtf?”

  4. @peeplaja Typical scenario: A company runs 100 tests / year. After 12 months their conversion rate is still the same.

  5. @peeplaja Testing mistake #1: Precious time wasted on stupid tests

  6. @peeplaja Testing mistake #2: You don’t have a data-driven, learning-oriented hypothesis

  7. @peeplaja Testing mistake #3: You think you know what will work

  8. @peeplaja Testing mistake #4: You run tests when you have no traffic

  9. @peeplaja Testing mistake #5: Your absolute conversions are way too low

  10. @peeplaja You need *at least* 250-400 conversions PER variation. Way more if you want to segment the data

  11. @peeplaja Our "thumb rule" is: 3000-4000 conversions per variation and 3-4 week test duration. That is enough traffic so we can even talk about valid data if we drill down into segments. Testing "sin" nr.1: search in segments for uplifts although you have no statistical validity, e.g. 85 versus 97 conversions. That’s bullshit.

  12. @peeplaja Testing mistake #6: Your tests don’t run long enough

  13. @peeplaja Testing mistake #7: You don’t test full weeks at a time

  14. @peeplaja Testing mistake #8: Test data is not sent to third-party analytics

  15. @peeplaja Pro tip: If you have over 500,000 pageviews / mo, you don’t (necessarily) need to spend $100,000 on GA Premium. Get unsampled data with Analytics Canvas

  16. @peeplaja Testing mistake #9: You give up after your first test for a hypothesis fails.

  17. 79.3% uplift in conversions Full case study: http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/

  18. @peeplaja Testing mistake #11: You’re not aware of validity threats

  19. @peeplaja History effect Instrumentation effect Selection effect

  20. @peeplaja Testing mistake #12: You’re ignoring small gains

  21. @peeplaja 5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how math works.

  22. @peeplaja Testing mistake #13: You don’t know html/css/jquery - and thus only test the simple stuff

  23. @peeplaja Testing mistake #14: You’re not running tests at all times

  24. @peeplaja Testing mistake #15: You’re late for your lunch break!

  25. @peeplaja Thank you Keep reading ConversionXL! Follow me @peeplaja

More Related