A closer look at Testing Hans Axelsson The view of testing in this presentation is that of my own and doesn’t necessarily coincide with any official company policy. I have been inspired by James Bach, Michael Bolton and James Whittaker, together with a whole lot of blood, sweat and tears.
Bwin Games • Started in Vienna 1999 • The worlds largest betting company. • Bwin has over 20 million registered customers in more than 25 core markets. • Bwin bought Ongame AB in 2006 • Bwin is one of Ongame’s largest customers 9/1/2014 Ongame.com 4
Ongame • Started in Uppsala 1999 • Owns the Ongame network. • Ongame is owned by Bwin Games 9/1/2014 Ongame.com 6
Ongame A question to the audience: Have you ever encountered bugs in commercial software?
What is testing? 9/1/2014 Ongame.com 8
What is testing? 9/1/2014 Ongame.com 9
What is testing? 9/1/2014 Basic facts about software development: • For every 10-percent increase in problem complexity, there is a 100-percent increase in the software solution’s complexity. (Robert L. Glass) • The most important factor in software work is the quality of the programmers. (Jeff Atwood) • Humans make mistakes. (Common sense) • Software Errors Cost U.S. Economy $59.5 Billion Annually (NIST 2002) Ongame.com 10
What is testing? Error types and their origin * * (From “Reducing error and influencing behaviour”, HSE Books) 9/1/2014 Ongame.com 11
What is testing? • A technichal investigation for the purpose of revealing the quality of a software product on behalf of the stake holders. (Cem Kaner) • Try it and see if it works(Basic conception?) 9/1/2014 • Questioning a product in order to evaluate it. (James Bach) Ongame.com 12
What is testing? Try it Get different versionsSet them upTry simple thingsTry complex thingsTry sequencesTry combinationsTry weird thingsTry them again and see if Where to look?How to look?What‘s there?What‘s not thereWhat‘s invisible?Did it change? Will it change? How about now? Read the spec(There‘s no spec? Oh.)(There is a spec! Oh it‘s old and wrong)Find inconsistenceisFind obvious problemsFind obcsure problemsFind BAD problems it works 9/1/2014 Ongame.com 13
What is testing? So, testing is (for us) basically about: „Try to learn sufficiently, everything that matters about whether it canwork and how it might notwork.“
Testing as a profession What is typical for a tester? Curious: A tester wants to know about the domain he/she operates in. If there is something that cannot be understood, a good tester is quick to learn and hungry for knowledge. This way of thinking leads many testers to be good in a lot of different areas. They are often Jack of all trades.
Testing as a profession What is typical for a tester? Organized: The software development cycle is often tightly time driven. Must keep track of how to reproduce the abnormalities encountered. Have to keep track of the baseline versions where problems are found as well as fixed.
Testing as a profession What is typical for a tester? Skeptical:Question everything, but choose your main areas of attack. A tester knows there is at least one way of crashing an application or part of it. It’s just a matter of finding it. The knowledge that people are fallible gives the tester hope of finding bugs.
Testing as a profession What is typical for a tester? People SkillsCommunication is the most important tool for making changes. Testing is many times about showing that something DOESN’T work. The tester often have the bad news. Reporting bugs is not (always )the same as telling developers they are doing a bad job.
Testing as a profession How is a day af the office?
Testing as a profession Setting up systems (Configuration and installation) • Populating data (Generating or deploying a fixed setup) • Modeling and preparing tests (Tactics) • Running tests (Either small series or long integrated chains) • Looking at the right places (Logs, graphs, databases) • Evaluating the result (Comparing and discussing) • Communicating changes, deviations or suspect observations.
Working for Ongame/Bwin Testing is an intellectually very stimulating work. Testers are often developing a broad knowledge of their domain.
Working for Ongame/Bwin Testers question the software, but they must not forget to also question their own methods and ways of working. Tester are sometimes treated as bad messengers, often this is a defence mechanism when someone feels guilty or gets under a lot of pressure. It is essential for a tester to understand this.
Working for Ongame/Bwin Testing as a discipline is growing bigger and bigger. For anyone interested in test, we recommend James Bach’s site http://satisfice.com/
Working for Ongame/Bwin • Bonuses • Excellent private health insurance • Free breakfasts • Yearly discount gym card • Great VPN Solution • Lantern Day for individual development • Top Conferences to keep up on methods and techniques 9/1/2014 Ongame.com 26