CS5123 Software Validation and Quality Assurance

1. CS5123 Software Validation and Quality Assurance Lecture 9 Bug Report Management

2. Issue Tracking System After software testing Testers find a lot of bugs They need to tell the developers about the bugs They usually put them into a so-called issue tracking system Compared with Mailing list Hard to manage, come with all other mails Not categorized by topic No features for describing specific aspects in SE (versions, components, commits, etc) 2

3. Issue Tracking System A platform for developers to communicate with each other Like a forum People can register and raise a issue The one who raise the issue will describe the issue in details Others can comment 3

4. Issue Tracking System What is special More structured for describing issues Component, assignees, schedules, status, resolution Customizable Change the contents while progress is made (status, resolution) Sometimes allow anonymous issue raising Users of software are involved They use the software and raise bugs 4

5. Issue: an example 5

6. Issues – Life Cycle Submission Reporter Re-open Triaging Triager Duplicate/Invalid Closed Retriage Assigned Not reproducible Inspecting Developer Fixing Refix Fixed Verifying Developer 6

7. Issues Type of issues Process of issues Resolution of issues 7

8. Type of issues Bug reports Feature Requests Patches 8

9. Type of Issues Bug report (e.g., system shows wrong message) About a bug of the software Raised by a user/developer Should include: Step to reproduce Expected behavior Actual behavior Stack trace or error message if any 9

10. Example of a bug report https://bugzilla.mozilla.org/show_bug.cgi?id=876495 10

11. Type of Issues New feature (e.g., add sorting to results) About add a new feature, e.g., add sorting by modify time Raised by a user/developer 11

12. Example of a feature request https://bugzilla.mozilla.org/show_bug.cgi?id=382491 12

13. Type of Issues Patch (e.g., add checking for input validity) A fix to the software By a professional user or a developer on a important fix Should include: Version to patch Patch code: basically a code diff, Diff (buggy, correct) 13

14. Example of a patch https://bugzilla.mozilla.org/show_bug.cgi?id=657763 14

15. Process of an issue Open/New The issue is raised Nobody in the project has looked at it yet Assigned A person called triager assign issue to a developer Bug report: the developer will first reproduce the bug, and then try to fix Feature request: discuss with colleagues on whether to accommodate the request, and implement the feature Patch: Validate the patch Milestone: may be assigned to a sub-group 15

16. Process of an issue Closed When the decision on the issue is made in any way Fixed, usually accompany with code commits Reject Later Reopened After the issues is closed, something happens and the issue become active again Incomplete fix Start to implement a postponed feature Or revoke any wrong decision before 16

17. Resolution of an issue Fixed A bug is fixed A feature is added A patch is applied Invalid Bug cannot be reproduce Feature does not make sense (request is not understandable) Patch is not correct 17

18. Example of invalid bug reports https://bugzilla.mozilla.org/show_bug.cgi?id=198057 18

19. Resolution of an issue Duplicate The issue is a duplicate of another existing issue Often happens for user raised issues Usually bug report / feature request Some issue tracking system allows merge of duplicate issues Won’t fix The developers decide to not fix the bug or accommodate the new feature Limited human resource, new version is about to released 19

20. Example of duplicate bug reports https://bugzilla.mozilla.org/show_bug.cgi?id=137325 20

21. Example of won’t fix bug reports https://bugzilla.mozilla.org/show_bug.cgi?id=143329 https://bugzilla.mozilla.org/show_bug.cgi?id=290131 21

22. Existing Issue Tracking Systems Many project hosting websites provide issue tracking systems also Google Code: so you will also have a issue tracking system GIT Hub Source Forge Issue tracking service provider: Bugzilla Companies often has their own issue tracking system Users can submit issues, but the tracking system is not public 22

23. Tips on submitting an issue report Search for existing reports Specify: product, platform/OS, version Describe Input / Steps to reproduce Expected results / actual results Bug: Always reproducible or happen randomly (maybe related to concurrency, system resource, environ, etc.) Provide Snapshot / bug: error message Bug: Stack trace 23

24. Submit a bug report Demo on sourceforge, bugzilla, and github https://sourceforge.net/p/sweethome3d/bugs/new/ https://bugzilla.mozilla.org/enter_bug.cgi https://github.com 24

25. What are the most important part in a bug report? Step to reproduce? Test cases? Screenshots? Error Messages? Stack Traces? Version? … 25

26. A survey did by a number of German Scientists With 130 developers and 215 reporters Ask about what are the information they provided / used and expect to be useful Information types include: steps to reproduce, stack traces, test cases, observed behavior, screenshots, code examples, summary, version,… 26

27. Answer of the developers Step to reproduce: 83% Stack trace: 57% Test cases: 51% Observed Behavior: 33% Screenshots: 26% Expected behavior: 22% Code examples: 14% Summary: 13% Version: 12% Error report: 12% 27

28. Mismatch of understanding 28

29. Tips on triaging and handling issue reports Triaging: Search for existing reports Talk with the developers who are familiar with the area Decide the severity and who should handle the report (according to the expertise and workload) Handling an issues report (e.g., fix, add feature) Reproduce For features: communicate on the issue tracking system about whether and how to support it If reproducible Always report progress as comments in the issue tracking system: others may help or change their work accordingly 29

30. Review of issue tracking system Where the bugs go to after you find them Type of issues Resolution of issues How to write a good bug report? How bug reports are handled? 30