CSC407 Web Testing

1. CSC407Web Testing Greg Wilson gvwilson@cs.utoronto.ca Fall 2006

2. Testing • Remember testing? • It’s ironic: distributed applications are harder to debug, but we test them less • Standard web applications are not harder to test than other programs • Provided you design them for testability in the first place • The things you do to make them testable are things you should be doing anyway

3. Legacy Code • Legacy code (n.): source code that relates to a no-longer supported computer system • Often used as an insult, meaning “a mess you’d wish on your worst enemy” • Feathers’ definition: “legacy code is code that doesn’t have unit tests” • Without comprehensive unit tests, you can never be sure that your changes haven’t broken something

4. The Problem • How do you test this? PostgreSQL HTTP request Apache Python CGI Filesystem HTML

5. Naïve Strategy: Input and Output PostgreSQL HTTP request • Accept the system as given • Send HTTP, parse HTML • Faithful • Lots of work • And remember, screen scraping is fragile Apache Python CGI Filesystem HTML

6. Record and Playback • Do regression testing to make sure that we haven’t broken anything that used to work • So: • Record a working system (HTTP in, text out) • Replay the HTTP, and diff the result • WinRunner is the gold standard for desktop apps • MaxQ (maxq.tigris.org) • Allows non-programmers to create and run tests • But you have to have a nearly-working system… • …and it’s still screen-scraping

7. How About Changing the Problem? • Don’t look at HTML directly • Instead, insert markers and look for those • Much less likely to change over time <p> User ID: <em class=“test” role=“uid”>GVW</em> </p> • Going to use CSS to style the page anyway • Not checking everything… • …but giving yourself more time to check the things that require human eyeballs

8. …Changing the Problem PostgreSQL HTTP request Apache Python CGI Filesystem XML

9. Faking the Server • How about getting rid of the web server? • Run the CGI ourselves • Tell it to produce XML (or plain text) rather than HTML • Makes debugging easier • But testing even less faithful PostgreSQL FakeCGI Python CGI Test Code Filesystem

10. …Faking the Server • "All" we need is our own cgi.FieldStorage • And os.environ • And sys.stdin and sys.stdout • Design for testability • Code to interfaces, not implementations if testing: import test.FieldStorage as FieldStorage import test.environ as environ else: import cgi.FieldStorage as FieldStorage import os.environ as environ

11. More Practical Issues • Database is persistent storage • That’s its whole purpose • But tests are supposed to be independent • In theory, create a new database for each • In practice, just wipe and re-fill the tables • If tests are slow, programmers won’t run them

12. …Practical Issues • “If tests are slow” covers a lot of sins • How long does it take developers to set up? • Install the software the first time? • Reinstall/reconfigure for testing? • Will it interrupt real service? • Will it impact other developers? • SelectAccess: could take two full days to install and configure • Which was eight days less than the competition

13. …Practical Issues • Solution: change what you’re testing • SQLite and HSQLDB are much smaller than PostgreSQL • And both can store database in memory • Useless for real work, but great for testing • Design for testing: • Abstract away the details of external packages • A good idea anyway • Run developer tests using replacements • Still do final tests with real system, of course

14. Mock Objects • If no replacement is available, build a mock object • Behaves like a limited version of the real thing • E.g., when asked for contents of a file, hands back the path, reversed • Only works if you have a clean interface between your application and the subsystem • But you should anyway

15. A More Testable System SQLite-M • Much simpler to (re)install and (re)initialize • And much less likely to mess up production system • As faithful as you make it • Weeding out the easy problems gives developers more time to look at the hard ones FakeCGI Python CGI Test Code MockFS

16. Putting It All Together • Goal is to produce something that can: • Be run from the command line, so that its output can be compared to saved output (testing via Makefile) • Be called inside unittest (Python’s equivalent of JUnit) as part of a larger test suite • Would like to avoid creating temporary files • May need to pass command-line parameters to the thing being tested

17. A Week of Hard Work… • …can sometimes save 5 minutes of Google • HttpUnit • Java library that acts like a browser • Handles cookies, redirects, etc. • HtmlUnit • Models the returned page, rather than the HTTP transaction • Compatible with JUnit • There are lots of others… • For example, check out Selenium

18. Testing Summary • When testing a large system, ask: • Where and how will I inject my tests? • When and how will I check the results? • Every interface is a potential cut point • I/O, class creation, method calls… • The more you replace: • The easier testing is • But the less faithful it becomes • An economic decision

19. Exercise 2 • Design testability into your group signup system • “We’ll write unit tests” is not enough • What extra classes will be created (if any)? • How will they be configured into the system? • Etc.