Battle of Botcraft: Fighting Bots in Online Games with Human Observational Proofs

1. Battle of Botcraft:Fighting Bots in Online Games with Human Observational Proofs Steven Gianvecchio, Zhenyu Wu, Mengjun Xie, and Haining Wang

2. Outline • Background • Game Playing Characterization • HOP System • Experiments • Limitations • Conclusion

3. Outline • Background • Game Playing Characterization • HOP System • Experiments • Limitations • Conclusion

4. Background Online Games • In 2008, online game revenues $7.6B • about half from massively multiplayer online games (MMOGs) ex. World of Warcraft (WoW) • MMOG currency trades for real currency • players can make real money • A major problem is cheating

5. Background Game Bots • A common cheat is use of game bots • able to amass game currency • cause hyper-inflation • To combat bots • process monitors, ex. Warden for WoW • human interactive proofs (HIPs) • legal action

6. Background Game Bots • Glider – a popular WoW bot • controls game via mouse / keyboard APIs • uses profiles, i.e., configurations and waypoints • able to evade Warden • Blizzard sued MDY (maker of Glider) • awarded $6.5M

7. Outline • Background • Game Playing Characterization • HOP System • Experiments • Limitations • Conclusion

8. Game Playing Characterization Input Data Collection • World of Warcraft game • RUI program (with modifications) • records user-input events • converts events to user-input actions ex. move + move + press + release = point-and-click • computes user-input action statistics

9. Game Playing Characterization Game Bot • 10 Glider profiles (configurations and waypoints) • 40 hours • half with warrior and half with mage • levels 1 to mid-30s

10. Game Playing Characterization Human • 30 humans • 55 hours

11. Human • well fit by Pareto distribution • Game Bot • more fast keystrokes • signs of periodic timing Keystroke Inter-arrival Time Distribution

12. Human • fewer very short keystrokes • 3.9% shorter than .12 secs • Game Bot • 36.9% shorter than .12 secs • more signs of periodic timing Keystroke Duration Distribution

13. Human • highly-variable speed at all displacements • Game Bot • linear speed increases • high-speed moves with zero displacment Point-and-Click Speed vs. Displacement

14. Human • decays exponentially • only 14.1% of movements have 1.0 efficiency • Game Bot • 81.7% of movements have 1.0 efficiency Point-and-Click / Drag-and-Drop Movement Efficiency

15. Game Bot • no correlation between speed and direction Average Velocity for Point-and-Click

16. Human • diagonal, symmetric, and bounded • diagonals faster than horizontal / vertical Average Velocity for Point-and-Click

17. Outline • Background • Game Playing Characterization • HOP System • Experiments • Limitations • Conclusion

18. HOP System • A behavioral approach • human observational proofs (HOPs) • The idea: certain tasks are difficult for a bots to perform like a human • passively observe differences • HOP-based game bot defense system • continuous monitoring • transparent to users

19. HOP System • Client-Side Exporter • transmits user-input actions • Server-Side Analyzer • processes and decides: bot or human

20. HOP System Neural Network • Inputs 1. duration 2. distance 3. displacement 4. move efficiency 5. speed 6. angle 7. virtual key # of inputs = # of actions * 7

21. HOP System Neural Network • Output – human or bot Decision Maker • “Votes” on series of outputs ex. {bot + bot + human} = bot

22. Outline • Background • Game Playing Characterization • HOP System • Experiments • Limitations • Conclusion

23. Experiments Experimental Setup • 30 human players, 55 hours • 10 Glider profiles, 40 hours • 10-fold cross validation • test on a bot or human not in training set • 10 different training sets

24. Experiments HOP System • # of actions (input to neural network) • # of nodes (in neural network) • threshold x (on neural network output) > x is bot, <= x is human • # of outputs per decision ex. {bot + bot + human} = bot

25. Experiments Configure 1. # of actions and2. # of nodes • 4 actions with 40 nodes TPR and TNR vs. # of Nodes and # of Accumulated Actions

26. Experiments Configure 3. threshold and4. # of outputs • threshold 0.75 with 9 outputs per decision TPR and TNR vs. Threshold and # of Accumulated Outputs

27. Experiments Detection Results • Configured System • 4 actions, 40 nodes, threshold 0.75, 9 outputs • Glider – avg. true positive rate of 0.998 • Humans – true negative rate of 1.000 True Positive Rates for Bots

28. Experiments Decision Time • # of action * time per action • avg. 39.60 seconds Decision Time Distribution

29. Experiments Detection of Other Game Bots • MMBot in Diablo 2 • different bot, different game • without retraining the neural network • MMBot – true positive rate of 0.864 • Humans – true negative rate of 1.000

30. Outline • Background • Game Playing Characterization • HOP System • Experiments • Limitations • Conclusion

31. Limitations Experimental Limitations • Size • 30 not enough • Lab vs. Home • mostly in-lab • Character equipment / levels • Other bots and games

32. Limitations (cont.) Potential Evasion • Interfere with client-side exporter • block user-input stream • manipulate user-input stream • Mimic human behavior • replay attacks • model human user-input

33. Conclusion • Game Play Characterization • 95 hours of user-input traces • bots behave differently than humans • HOP System • exploits behavioral differences • compared to HIPs, HOPs are transparent and continuous • detects 99% of bots with no false positives • raises the bar for attacks

34. Questions? Thank You!

35. Questions? Thank You!

36. Questions? Thank You!

37. Experiments System Overhead • Memory • per user = 4 actions * 16 bytes + 16 outputs * 1 bit = 66B • server with 5,000 users = 330KB • CPU – P4 Xeon 3.0Ghz • 95 hours of traces in 385ms = ~296 hours/sec • server with 5,000 users = ~1.4 hours/sec