Object Oriented Analysis & Design

1. Object Oriented Analysis & Design Testing

2. Contents • Testing • Unit testing • CppTest • Regression testing • Equivalence testing • White & black box testing • Test coverage • Test plans • Test roles

3. Testing • Testing seeks to • Determine if the game meets requirements • If the game works as specified • If the game is bug-free • We test using • Unit tests • Tests by playing the game

4. Importance of Testing • Only high-quality games will sell • Players do not like bugs in games • It is difficult to get a program correct • Only testing can determine if a program is correct • Testing is an important part of producing a game!

5. Unit Testing • Unit Testing • Tests each class • Tests every method on every class • Unit testing shows that the classes work before they are put into a complete game • Unit tests are written as the classes are developed

6. Test Harness • Unit testing involves • Writing code to perform the tests • Checking the results • Reporting the results • There are frameworks which can aid with • A template to write the tests • An easy way to check results • Automating the reporting of results

7. CppTest • CppTest is • A lightweight unit testing framework for C/C++ programs • It can be downloaded from • http://cpptest.sourceforge.net

8. A Test • A test is • A void method with no parameters • Contains code to calculate a result • Uses a test assertion to compare the calculated result with the expected result • If the result is incorrect, the assertion will report the bug

9. Test Suites • A test suite is • A class derived from Test::Suite • Contains one or more test methods or other test suites • Adds each of the tests to the suite in the constructor • Can be run by calling the run() method which will run all tests in the suite

10. Assertions • TEST_ASSERT(boolean_expr) • Boolean expression is true to pass test • TEST_ASSERT_MSG(bool_expr, msg) • If test fails, msg is printed • TEST_ASSERT_DELTA(a, b, delta) • Test fails if a is not equal to B +/- delta • TEST_ASSERT_DELTA_MSG(a, b, delta, msg) • If test fails, msg is printed

11. Sample Test Suite #include <cpptest.h> class MathTestSuite: public Test::Suite { public: MathTestSuite() { TEST_ADD(MathTestSuite::addTest) } private: virtual void setup() { /* init code before first test */ } virtual void tear_down() { /* cleanup code after last test */ } void addTest() { TEST_ASSERT_MSG(2 + 2, 4, “this does not add up”); } };

12. Running a Test Suite int main(intargc, char **argv) { MathTestSuitemts; Test::TextOutput output(Test::TextOutput::Verbose); mts.run(output, true); // true runs rest of tests after a // test fails }

13. Test Guidelines • Each test should test one simple operation • Each test should contain one assertion • Related tests should be in the same test suite • Related test suites can be combined into a test suite using the add(Test::Suite) method of the test suite

14. Regression Testing • Once a test suite is created, it can be easily run again and again • This means we can run the test suite • After every change to the code • Before the changes are checked into the repository • Before you ship a release

15. Selecting Test Cases • It is impossible to test all possible values • We identify groups of values which test the same thing • Then we test only one value from each equivalence group

16. Equivalence Testing • Most bugs in programs occur at the boundaries of the data or on special values • Zero is a special value • MAXINT and MININT are boundaries • An empty string and the longest permisable string are boundaries • A NULL string is a special value

17. Equivalence Testing • Test the boundary values • Test values adjacent to the boundary values • Test values just outside the boundaries to see if the code handles these correctly • Test a few routine values to make sure they work • Test values that should cause the code to fail

18. Sample Equivalence Tests • You need to test a function which takes a single integer parameter greater that zero • Use the following test data • Largest negative value • -100 random negative value • -1 near boundary • 0 boundary • +1 near boundary • 8 routine even value • 9 routine odd value • Largest positive value -1 • Largest positive value

19. Black Box Testing • Treats the software as a black box which you do not know how it works • Tests that the documented features work correctly • The black box tests can be designed as soon as the requirements are available

20. White Box Testing • Tests designed based on a knowledge of the internals of the game • Must be designed by a programmer • Creates tests which • Test the path through every if and switch • Test unusual error conditions • Test infrequently executed code • Invoke specific methods

21. Testing Coverage • Most testing leaves many parts of the code unexecuted • A coverage analyzer can tell you the number of times each statement was executed • This can identify untested code • You can design white box tests to test the untested code

22. Game Testing • When most people speak of game testing, they mean functionality testing • This involves • Performing every operation • Performing every combination of operations • Executing every menu item • Executing every level • Ensuring the results are as expected

23. Functionality Testing • Functionality testing is performed by • Creating a test plan which • Specifies how to set up each test • Specifies the steps in conducting each test • Specifies the expected results • The game tester is responsible for • Executing each of the tests • Reporting any bugs found

24. Compliance Testing • Special agreements are imposed by • Console manufacturers • Software / hardware vendors • Governmental agencies • Entertainment Software Rating Board • Compliance testing involves checking that the game does not violate any of these agreements

25. Compatibility Testing • Games need to test for compatibility with • Different versions of the operating system • Different versions of graphics libraries • Different versions of third party libraries • Different input devices (mice & joysticks) • Different graphics cards • Processors with different numbers of cores • The minimal hardware the game requires

26. Localization Testing • Most games are sold in international markets and have to be correct in many languages • Check for • Is the locale automatically detected ? • Are all menus translated correctly ? • Is printed dialogue translated correctly ? • Is audio correct ? • Are number, date and moetary formats correct? • Are there offensive cultural references?

27. Cultural Considerations • What is good in one culture is bad in another • Some numbers are bad luck in some cultures • Any hand gesture is obscene somewhere • Colours have different emotional connotations in different cultures • Cultures have different attitudes towards • Men and women • Religion • Authority • Short term versus long term thinking

28. Soak Testing • This involves leaving the game running for long periods of time in different modes • Can also involve performing the same operation a large number of times • Can detect • Memory leaks • Buffer overflows • Race conditions • Huge output files

29. Load Testing • This puts a high load on the system to ensure it will not fail • A large number of players on an MMO • A large number of enemy characters • A large number of threads • Fast input over extended time periods • Large amounts of scenery

30. Multiplayer Testing • Tests to see that the program works well with multiple players • Supports large numbers of players • Works over various network types • Has good performance under high network load • There is acceptable lag in updating the view of each player • Single player mode works as well as multiplayer

31. Installation Testing • Does the program install correctly on all target systems ? • Are the correct resources installed for each locale ? • Is the installer in the right language ? • Does the documentation agree with the version of the game it shipped with ? • Can users understand the documentation ?

32. The Test Plan • The test plan • Provides details of all the test to conduct • For each test it specifies • How to prepare for the test • The steps in performing the test • How to determine if the test passed or failed

33. The Test Report • Produced after conducting a test • States • The test conducted & time & date • Any special conditions for the test • The result(s) of the test • If the test failed • All available logs from the game when the test failed • Observations by the tester

34. Roles in Testing • Game producer • Sets testing deadlines in conjunction with marketing • Lead tester • Manages the testing process • Keeps records on the tests performed • Tracks bugs reported • Understands how to program

35. Roles in Testing • Test Designer • Responsible for designing the tests • Usually a programmer who knows the type of things which go wrong • Tester • The person who actually conducts the tests • Does not require programming skills • Must be detail oriented • Must like repetitive tasks

36. Beta Testing • After testing within the company is complete, the game is released to select customers • They get the game early with the agreement that they will • Play the game a lot • Report any bugs they find • Send in logs produced at the time the bugs occurred