Optimization Success Story: Match’s Cultural Change

1. Optimization Success Story: Match.com’s Cultural Change Jim McDonald, Sr. Online Marketing Manager, Match.com

2. Overview • Match.com’s Culture Change • Told from a Relationship /Dating Angle

3. Frequency Down A/B Testing once every two, three weeks – sometimes not for months! Friends & Colleagues bragging about doing it all the time! Why Relationships Grow Stale

4. Frequency Down Staying Home Up Use internal A/B testing platform, data & analytics Bored with routine Results never seem to change Want more Scientific Approach Anecdotes and gut feeling are often used Rank determines test Don’t want to be seen in public Never have anything new or worthwhile to share Why Relationships Grow Stale

5. Frequency Down Staying Home Up Playing Darts Up Throw darts to determine next A/B versions to test Why Relationships Grow Stale

6. Frequency Down Staying Home Up Playing Darts Up Complacency Up People do Not like Change! People know what to expect with the Status Quo Why Relationships Grow Stale

7. Why Think about Change • Things Getting Old • A/B tests take time to test and iterate • Seems like you’re spinning your wheels • Takes forever to commit

8. Why Think about Change • Things Getting Old • New Person Enters Picture • Someone catches your eye • You learn from someone else • New Colleague, Director who as used MV testing • Someone questions the status quo • You catch someone else’s eye • Someone else learns from you • You bring a new paradigm to your company/department • You questions the Status Quo

9. Why Think about Change • Things Getting Old • New Person Enters Picture • Lost Trust • Data behaving strangely • Not totally confident in results

10. Brings Back Passion What are the Benefits of Meeting Someone New? • Tests are Easier • Tests are Quicker • Results Baby! • You want to be seen

11. Brings Back Passion Brings Back Trust What are the Benefits of Meeting Someone New? • No more excuses • You don’t question the results • You know what is affecting change • Sector Variable reporting • No more throwing darts • No more design or test by committee

12. You’ve Seen the Light Like what you see Everything makes sense Everything seems certain and conclusive Sometimes the grass IS greener Sometimes you need an independent sound-check Why Break Ties with Your Old Relationship?

13. You’ve Seen the Light You have More Self Esteem Know you can do better Good is not good enough where better is expected Why Break Ties with Your Old Relationship?

14. You’ve Seen the Light You have More Self Esteem The Data Proves Itself Gives you definition You know specifically what works and what doesn’t Gives you direction no more darts, no more design by committee No more unanswered questions Why Break Ties with Your Old Relationship?

15. You Do it all the Time Now MV Testing is Iterative MV Testing is Scalable Touches every facet of an online product Not just going through the motions MV Testing produces results! What Happens in the New Relationship?

16. You Do it all the Time Now You Want to Be Seen You Look REALLY Good! Results speak for themselves You ARE Affecting Change! You Want People to Talk to You You want people to ask questions Real knowledge is power What Happens in the New Relationship?

17. You Do it all the Time Now You Want to Be Seen Become an Ambassador Share your Knowledge Communicate Benefits with Other Departments Tell them it’s FREE for them Insure a Higher Power is Involved Your Boss, Your Bosses Boss… What Happens in the New Relationship?

18. Doing it all the time Now You Want to Be Seen Become an Ambassador Share the Benefits SPEED! DIRECTION! Reporting Data is easily understood Simplicity My marketing assistant and I implement all code (three snippets) Optimost generates code for us and sets up the campaign What Happens in the New Relationship?

19. Two Streams Big Idea Stream Looking for large incremental lifts Do A/B Testing Promote if beats Control (current best performer) to significance M/V Stream Test derivations of best-performer Constantly promote best performer at significance Testing Methodology – How We Test

20. Testing Methodology - Testing Example

21. Within First Month 10% lift in subscriptions One minor tweak on landing page Recently Another 10% incremental lift in subscriptions Another minor tweak Approximately 3% additional lift from other combinations 10% reduction in marketing costs Testing Methodology - Results

22. Final Thoughts • MV Testing = Easy • Much easier than going through a development prop cycles • Gives you control over iterative changes • MV Testing = Inexpensive • What does a 1, 2, 5, 10% incremental lift with add to your revenue • What does developer and analyst time cost with traditional internal A/B systems • Monthly fee is relatively nominal – worth testing • MV Testing = FUN! • Optimost makes my job fun • Like playing a game and getting paid for it • Multivariate Testing = Results • Subtle results all add up • Dramatic results will be uncovered • You WILL be surprised with the results

23. Questions? • Jim McDonald • Sr. Online Marketing Mgr. • Match.com • jimhmcdonald@gmail.com • 214-597-1301 “A website that can’t be found is useless… …A website that can’t be used is pointless”