1 / 27

CS 3773 Software Engineering

CS 3773 Software Engineering. Basic Concepts of Software Testing Unit Testing . Why testing? Examples. Mars Climate Orbiter ($165M, 1998) Sent to Mars to relay signal from the Mars Lander Smashed to the planet: failing to convert between English measure to metric values

rusty
Télécharger la présentation

CS 3773 Software Engineering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 3773 Software Engineering Basic Concepts of Software Testing Unit Testing

  2. Why testing? Examples Mars Climate Orbiter ($165M, 1998) Sent to Mars to relay signal from the Mars Lander Smashed to the planet: failing to convert between English measure to metric values Shooting down of Airbus A300 (290 death, 1988) US CG-49 shoot down a Airbus A300 Misleading output of the tracking software THERAC-25 Radiation Therapy (1985) 2 cancer patients at East Texas Cancer center received fatal overdoses Miss-handling of race condition of the software in the equipment 2

  3. Why testing? Numbers On average, 1-5 bugs per KLOC (thousand lines of code) In mature software More than 10 bugs in prototypes Windows2000 35MLOC 63K known bugs at the time of release 2 bugs per KLOC $59.5B loss due to bugs in US 2002 (estimation by NIST) It is not feasible to remove all bugs But try to reduce critical bugs 3

  4. Answer is testing, why? “50% of my employees are testers, and the rest spends 50% of their time testing” ---- Bill Gates, 1995 More reliable than inspection, relatively cheap Actually in the old days, when testing is expensive, inspection was the major answer You get what you pay (linear rewards) Compared to other 3 approaches Inspection, static checking, formal proof 4

  5. Testing: Concepts Test case Test oracle Test suite Test script Test driver Test result Test coverage 5

  6. Testing: Concepts Test case An execution of the software with a given list of input values Include: Input values, sometimes fed in different steps Expected outputs Test oracle The expected outputs of software by feeding in a list of input values A part of test cases Hardest problem in auto-testing: test oracle problem 6

  7. Testing: Concepts Test suite A collection of test cases Usually these test cases share similar pre-requisites and configuration Usually can be run together in sequence Different test suites for different purposes Smoke test, Certain platforms, Certain feature, performance, … Test Script A script to run a sequence of test cases or a test suite automatically 7

  8. Testing: Concepts Test Driver A software framework that can load a collection of test cases or a test suite It can usually handle the configuration and comparison between expected outputs and actual outputs Test Coverage A measurement to evaluate how well the testing is done The measure can be based on multiple elements Code Input combinations Specifications 8

  9. Granularity of Testing: V-model 9

  10. Granularity of testing Unit Testing Test of a single module Integration Testing Test the interaction between modules System Testing Test the system as a whole, by developers on test cases Acceptance Testing Validate the system against user requirements, by customers with no formal test cases Regression Testing Test a new version with old test cases 10

  11. Unit testing Testing of an basic module of the software A function, a class, a component Typical problems revealed Local data structures Algorithms Boundary conditions Error handling 11

  12. An example: writing test cases without a test framework Class Definition Consider a class that handles and manages a Fibonacci array Initialized the class Fibonacci f = new Fibonacci (); Extend to a certain length f. extend (length) Provide the number at certain index f. get(index) Get Length f.getLength() Provide a range of numbers f. getRange(start, end) 12

  13. An example: writing automatic test cases without a test framework • public static void main(String args[]){ • Fibonacci f = new Fibonacci(); • if(f.getLenght != 0){ • System.err.print(“wrong results …”); • } • f.extend(50); • if(f.getLenght != 50){ • System.err.print(“wrong results …”); • } • int num = f.get(50); • if(num != 12586269025){ • System.err.print(“wrong results …”); • } • int[] range = f.getRange(40, 50) • if(!ValueEquals(range, …)){ • System.err.print(“wrong results …”); • } • … • } Not Enough! What happened if an exception is thrown 13

  14. An example: writing automatic test cases without a test framework • public static void main(String args[]){ • Fibonacci f = new Fibonacci(); • if(f.getLenght != 0){ • System.err.print(“wrong results …”); • } • f.extend(50); • if(f.getLenght != 50){ • System.err.print(“wrong results …”); • } • int num = f.get(50); • if(num != 12586269025){ • System.err.print(“wrong results …”); • } • int[] range = f.getRange(40, 50) • if(!ValueEquals(range, …)){ • System.err.print(“wrong results …”); • } • … • } Potential test interference! 14

  15. Common Operations Comparison between actual results and expected results Report error messages Catch Exceptions Re-initialize the class to be tested 15

  16. Unit test framework xUnit Created by Kent Beck in 1989 This is the same guy we mentioned in XP, design patterns The first one was sUnit (for smalltalk) Junit The most popular xUnit framework There are about 70 xUnit frameworks for corresponding languages Never in the annals of software engineering was so much owed by so many to so few lines of code -------Martin Fowler 16

  17. State machine of JUnit try{ testMethod } catch {…} Initialization Of Test Class setUp tearDown Why? Try and Catch? 17

  18. Writing a Test Case public class FibonacciTest extends TestCase { protected Fibonacci fTest; protected static int groudtruth[] = {0, 1,...,12586269025}; public void setUp(){ fTest = new Fibonacci(); } @Test public testInitialize(){ assertEquals(“Length of fTest after initialization”, 0, fTest.getLength()); } @Test public testExtend(){ fTest.extend(50); assertEquals(“…”, 50, fTest.getLength()); } @Test public testGet(){ fTest.extend(50); assertEquals(“…”, groudtruth[49], fTest.get(50)); } … } 18

  19. Benefits of using test framework Clear structure of testing Each test method for one feature Common initialization to setup Assertion methods You can always define new assertion methods Reduced Interference Try catch to run all test methods Always do re-initialization in setup 19

  20. Demo Writing a JUnit testcase for a class in your project Run it as a JUnit test case 20

  21. Tips of writing test cases Bottom Up If you know that obj.m1 -> obj.m2, test obj.m2 first Why? You may not understand the error in m1 if you don’t know whether m2 is working or not Externalize data Externalize expected results to files and load them at the test case constructor Why? Easier to maintain if you want to reuse them in several places 21

  22. Tips of writing test cases Use development Database Do not use real database for testing Why? Obvious Do enough logging in the code during testing Why? Help debugging Testing for exceptions How? public testException(){ try{ do(invalid); fail(); }catch(xxxException e){} } 22

  23. Assertions public void testCapacity() { // a test method …. assertTrue(fFull.size() == 100+size); //assertion } If assertion fails: Assertion failed: myTest.java:150 (expected true but was false) Not so good! Try: assertEquals(100+size, fFull.size()); //expected value first Assertion failed: myTest.java:150 (expected 102 but was 103) Better! Try: assertEquals(“list length”, 100+size, fFull.size()); Assertion failed: myTest.java:150 (list length expected 102 but was 103) 23

  24. Assertions Extend TestCase and write your own assertions AssertStringContains, AssertArrayEquals, … Use fail(String) to fail a test with certain message Feel free to change the fixture Each test will re-setup public void testRemoveAll() { fFull.removeAllElements(); fEmpty.removeAllElements(); assertTrue(fFull.isEmpty()); assertTrue(fEmpty.isEmpty()); } 24

  25. Tear down Consider the following test code void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { try { use f and b } finally { clean&close f, b } } void testBBB() { try { use f and b } finally { clean&close f, b } } void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { use f and b } void testBBB(){ use f and b } Better? Problems?

  26. Tear down • Consider the following test code void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { use f and b } void testBBB(){ use f and b } void tearDown{ clean&close f, b } void setUp() { File f = open(“foo”); File b = open(“bar”); } … void tearDown{ try{ clean&close f }catch{ … } the same for b } Problems? 26

  27. Tear down • Be careful about tear down • If tear down is not complete, a test failure may affect the following test cases • Recover the changes done to global data that are not well handled by the setup • Database, files, network, global variables • Clean resources • Caution of exceptions in tear down itself 27

More Related