1. Test Specification Sabesan

2. Why test? • Improve quality • Find bugs (no crashes, erroneous calc) • Verify the requirements? • To put it another way: • Testing is a verification process to improve the confidence that the system conforms to a specification • Your job is to “prove” that the specification is implemented correct

3. Fulfill requirements • Think of how to test each requirement • Some requirements should be considered in all test cases (e.g. write to output file) • Others have specific tests • Nothing unexpected (if spec. is not open)

4. Writing a test case • Numbered • Refers to requirement(s) • Prove valid test case (against spec) • Shows that all requirements are addressed (Suite) • Tip: A requirement that would have generated an error earlier does not need to be mentioned over and over again • Explains HOW to execute the test • For dummies, include all required info • Explains the in-parameters and the expected output. • User (or script) able to determine the verdict PASS or FAIL

5. Reference • The purpose of the Test Specification is to verify a system according to a referenced Requirement Specification, i.e., add VERSION number of the specification • Most of the formalism as RS

6. Test Specification • A. Part 2 • Test case A.1 • • Tested requirements: • - Req. A.3.10 *) (boundary testing: value inside the range) • - Req. A.3.14 *) (boundary testing: value inside the range) • - Req. A.3.15 *) (boundary testing: value inside the range) • - Req. A.1.27 • *) mainly targeted requirements for this test case • • Test data: • - Change to a directory where you have write permissions. For example: cd ~/ • - Create a simple XML-Input file containing a camera (location:[0,0,0] sky:[0,1,0] look_at:[0,0,1]) and a red sphere (radius: 1.2 location:[0,0,12] surface:pigment:color[1,0,0]) and name the file “simple_scene1.xml”. • - Write the following command to the shell: tracer -z 500 -x 500 -y 500 -w 1 -h 1 test_a1.ppm <simple_scene1.xml • - Write the following command to the shell to view the output picture: xview test_a1.ppm • • Expected result: • - The program should terminate without error message.

7. Tests for Non-Functional Req • How to “test” a req. “Code should be written in a nice and structured way”? • Refer to code review • Verdict PASS if code review PASS • If code not available write N/A “not applicable” • This should be in the Test Plan (not in TS) • How to “test” a req. “Use getopt lib” • Query supplier e.g. ask developer/manager • This is a simple YES/NO question

8. Testing limitations • A crashed program is “not an option”. • Parameter range e.g. if the specified range is [1..1000] this gives the tests • Inside range close to bound, i.e., 1 and1000 • Common or other value picked “random” inside the range, e.g., 100 • Outside range close to bound, i.e., 0 and 1001 WHAT will happen here…

9. Combinations • You cannot test all cases, e.g., 1000*1000*1000 values • Vary upper and lower bound • Vary object hidden each other (partly or fully) • Vary object not in view of the camera • You can discuss functionality with you MP2 group, e.g., how the camera works or how the XML file works, but NOT the test cases

10. Discussion • Is it good to slim the test suite?

11. Equivalent tests • For function “f(x)” • Test f=(0 if x=0),(x if 0<x<10), and (0 if x=10) • If range for x is [0..10] • f(0) f(some in [1..9]) f(10) must be tested • Classes {0} {1,...,9}{10} • Is f() affected by something else? • Can we suspect problems if something else changes?

12. Report

13. Report • Which system is tested (code version) • Who has tested • Which test specification used (version) • RS