1 / 33

Building a Web Recommendation and Customization Platform at Big Fish Company

Big Fish Company, the world's leading casual game producer, established a robust recommendation and customization platform by analyzing its extensive catalog of over 3,000 casual games. Founded in 2002, it has successfully distributed over 2 billion games globally. The project aimed to enhance sales and customer lifetime value through intelligent recommendations and tailored customer experiences. By leveraging advanced data solutions, including relational and NoSQL databases, the platform was designed to handle high traffic, ensure data consistency, and offer personalized game suggestions.

desma
Télécharger la présentation

Building a Web Recommendation and Customization Platform at Big Fish Company

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building a Web Recommendation and Customization Platform September 2013

  2. Big Fish Company Overview Big Fish is the world’s largest producer and distributor of casual games • Founded in 2002, Big Fish has distributed over 2 billion casual games and launchesA New Game Everyday!® for PC & Mac • 3,000+ games in the Big Fish catalog from 140+ exclusive game developer partners worldwide • Global business with games sold in 150+ countries • Over 100 million people have played games in the Mystery Case Files®franchise • Big Fish has published over 300+ unique mobile games on iOS, Android and other platforms • Offices located in Seattle, WA; Oakland, CA; and Luxembourg

  3. Our Distribution Catalog Our catalog of 3,000+ casual games on many platforms are distributed through our multiple channels

  4. Big Fish by the Numbers 2 billion+ Total PC/Mac games downloaded from Big Fish’s leading online marketplace 140+ Million Mobile games downloaded 100 million+ People who have played a Mystery Case Files® game (flagship brand) 10 million+ Unique visitors per month to Big Fish’s web portals 2 million+ Retail units sold annually (e.g. Wal-Mart, Target, Tesco and other large international retailers) 1 million+ Games distributed per day worldwide 3,000+ Unique games in the Big Fish catalog 600+ Game developer partners worldwide 300+ Unique games for smartphones and tablets 150+ Countries where customers purchase games by Big Fish 40+ Percentage of revenues outside North America 3 Offices (Seattle, WA; Oakland, CA; Luxembourg) 1 #1 PC/Mac casual game app store in the world

  5. Agenda • Business Requirements • Project Requirements • Data Solutions Evaluated • Evaluation Results • Technical Details and Implementation • Outcome and Results

  6. Game Recommendation System • Components: • Recommendation engine • Purchase and download tracking • Failsafe logic • Backfilling ‘light’ recommendations • Global recommendations • Service down fall-through Game Recommendation

  7. Business Requirements • Increase sales • Increase Customer Lifetime Value • Consistent results between platforms • Targeted game marketing • Never show a product the user cannot buy Game Recommendation

  8. Standard Homepage (Default)

  9. Intelligent Recommendations

  10. Big FishApplication Generic Recommendations (Not Logged In) Targeted Recommendations (Logged In) Game Recommendation

  11. Project Requirements • Ability to meet traffic demands for 3 years • Real-time purchase and download tracking • Updates to the customers recommendations automatically happen • Tracking of all algorithms and clicks • Able to rebalance shards without Engineering Game Recommendation

  12. Data Solutions Evaluated • Existing MySQL database • MySQL Master / Slave topology with 1.5BB row table • NoSQL • Redis • Hadoop • Riak • Membase • Membase + MySQL (hybrid) • Cassandra • Hbase Game Recommendation

  13. Evaluation Results General questions and concerns each solution • Data consistency • Disaster Recovery (Backups) • Failover • Performance • Persistence • Risk to move to a new system • Expertise • Cost Game Recommendation

  14. Evaluation Test - Metrics Lookup Directory Inserts and Lookups per Second Single Database Performance VS Sharded Database Performance Game Recommendation

  15. Sharding by the Numbers • New design utilized composite PK based on customer, game and language • Lookups were all primary key; very performant • Write load is light; do not suffer from mid-table inserts Current Numbers: • Database: 75 queries per second with 912 µs response time • Memcache: 600 requests per second with 185 µs response time • Database is over 100GB with 2.14BB+ records • Grows at a rate of 750,000+ records per day Game Recommendation

  16. Sharding Implementation • Lookup service database has 2 tables: • One for shard definitions • One for mapping the customer to the shard • Leverage 2 stored procedures for data integrity and acid compliance: • One locates which shard receives next customer • One gets customer shard information or creates it Game Recommendation

  17. Outcome and Results • From inception to code release ~ 6 months • Launched all backend services in Q4 2011 • Performed 6 iterations updating modules and pages on our site to utilize the system • Deployed recommendations to Big Fish Application Game Recommendation

  18. Content Customization System • Content: • Ability to change assets on the website per customer • Customization: • Leveraging customer data and behavioral patterns to create a personalized experience Content Customization

  19. Business Requirements • Increase revenue through customized experience • Admin tool for internal marketing teams • Ability to support 100% delta change per day • Integrate with all Big Fish products and services Content Customization

  20. Existing customer homepage

  21. New Customer Homepage

  22. No Campaign Triggered

  23. Campaign Triggered

  24. Project Requirements • Long term scalability • Support up to 250MM+ records (tested 750MM) • Performant system for reads and writes • Enable the business while reducing Engineering and Operations maintenance costs Content Customization

  25. Data Solutions Evaluated • Traditional replication deployment • Sharded replication deployment • Solves / provides workarounds for most problems • MySQL Cluster (NDB) • Ability to quickly load data and retain high read rates • Auto-sharding • In-memory solution; removed memcache layer • Reduced Engineering QA efforts • Enterprise support (Oracle technology, MCM toolset) Content Customization

  26. Evaluation Results • Ability to load ~180,000 records per second using 4 MySQL threads, batch size of 10,000 records • Insert 100MM in less than 10 minutes • Tuned Cluster to be write intensive resulting in 150,000 inserts/sec and 3,000 reads/sec • ~27,000 reads/sec Content Customization

  27. System Architecture http://www.mysql.com/why-mysql/case-studies/mysql-cs-bigfish.html

  28. NDB Connectors Reads Writes Content Customization

  29. Outcome and Results • From inception to code release ~ 4 months • Deployed Q2 2013 • MySQL Cluster supports 10-12M requests per day (1-1 mapping for page views) • Phased launch: • Support all International sites • Support US site Content Customization

  30. Overall Recommendation Platform Results • Enabled the business to increase revenue • Nominal maintenance (< 1%) • Five 9’s availability

  31. Questions?

  32. Thank You!

More Related