Sunday, March 22, 2009

Bug report: How to write an effective bug report?

View the Bug Reporting video or read on. Each one of us in the software testing field has seen many bug reports. Bug reports are written in a vast variety of styles. Sometimes, we happen to see the bug reports created by experts and inspire ourselves by doing so. However, a test engineer may create a bug report in any manner as seen fit. If so, can one be sure that the bug report is best suited to the circumstances? Will the bug report do the job well? In this article, I wish to make my reader aware of various aspects of a bug report. I have attempted to integrate many methods used by experts. This will enable you, my reader, to create bug reports that are best suited to the conditions at hand. The next time you have to write a bug report, you will have many tips to choose. Alternately, you may implement certain parts of some tips to create a custom strategy that suits the purpose.

How to write an effective bug report?

The first thing that we need to do is to understand the meaning of a bug report. Put simply, a bug report is a problem statement. It describes a problem observed in the system under test. Along with the problem, a number of supporting data elements is usually present. A bug report is used to communicate important problem data between testing, development, support and management groups. It is possible that the bug report created by the reader is read and acted upon in a different country and/ or in a different organization. All the currently open bug reports collectively contribute to indicate the present quality of the system.

Next, let us look at the various data elements that can exist in a bug report. Depending on the reporting requirements or your wishes, you can use the fields available in the defect tracking system to convey the bug to the intended audience clearly.

1. Id: Each bug should have a unique identifier. If a defect tracking system is being used, this Id is usually generated automatically. If not, you can assign a unique Id to the bug report. Since managers use bug Ids to track the bugs and the bug Ids are present in past reports, you should never alter the bug Ids.

2. Summary: The summary is a very important part of the bug report. Ideally, the summary should capture the core of the problem in a few words. A summary can contain other supporting information e.g. project/ product version, build, component, platform/ environment, team name, name of the bug reporter etc. You should consider the norms of writing bug reports in your project/ organization and summarize appropriately.

3. Project/ product: The bug reporter usually files the bug report against a single project or product in the system under test.

4. Component/ Module: If the bug belongs to a particular part of the system, you should name that part of the system. On the other hand, if it were not possible to attribute the bug to a particular part of the system, you may not name the Component/ Module or may term the bug as a “system-wide” bug.

5. URL: While testing web applications, it is common to state the URL that is required to retrace the steps and observe the problem.

6. Build: The Build number is the build (major number and/ or minor number) in which you discovered the problem.

7. Type of bug: The Type of bug can be a functional problem, a data problem, a performance problem, a problem with the build/ installer or a problem with the user interface.

8. Priority: Though you can set up the priority of the bug report based on the anticipated impact of the problem on the users, development or management may set the priority value differently, if required.

9. Severity: The Severity value is set by the bug reporter based on the problem itself e.g. it can be showstopper or blocker or a high severity bug or a medium severity bug or a low severity bug.

10. OS/ Platform/ Environment: If testing was done in different environments e.g. different operating systems (Windows XP, Vista, Linux etc.) or different browsers (IE, Firefox, Safari etc.), it is useful to specify the particular environment, in case the bug is observed in that environment only. For bugs that exist in all environments, you may mention the extent of the problem in the Description field.

11. Description: This is perhaps the most important part of the bug report. It contains the basic steps required to reproduce the problem. Sometimes, it is useful to mention both the expected results as well as the actual results encountered on execution of the steps. The actual results are the problem statement. In order to obtain support to get the bug fixed, it may be useful to state the impact of the actual results on the users. You may mention any data element that does not fit in the fields available in the defect tracking system in the Description part of the bug report under appropriate heading(s).

12. Attachments: A screen capture or a short video (consider the size of the attachment) showing the problem is usually quite effective in explaining the problem to the recipients of the bug report. This file should complement the Description of the bug report. If the file shows a number of items, encircling or otherwise highlighting the problem area in the file would save the recipients time to understand the problem. In addition to screenshots/ videos, you may attach other items e.g. error log, a report, other documents to build support for bug fixing.

13. Reporter: In many defect-tracking systems, this field is automatically populated. In case it is not, you should mention your name and contact information here.

14. Assigned To: You could keep any incomplete (or “in progress”) bug report assigned to yourself. A completed bug report should be assigned to the correct person in development (a development manager/ lead or a developer). A fixed bug report should be assigned to the bug reporter so that it may be re-tested and marked as closed. In case the assignee requires information or a decision from another stakeholder e.g. management or marketing, the assignee may assign the bug report to the concerned person in the relevant group. Note that at any particular time, the bug report should stay assigned to exactly one person.

15. Cc: Often, some persons are interested in the bug reports logged against the system under test. However, they may not play any direct role in fixing the bug. Such persons can include leads and management. These persons should stay informed about any changes to the bug report.

16. Dates: A single bug report can include a number of dates. Common dates included in the bug report include Date Created, Date Assigned, Date Resolved, Date Reopened, and Date Closed. In case the date fields are not automatically maintained by the defect tracking system, care should be taken to log the correct dates. A number of testing metrics e.g. bugs opened/ closed, average age of an open bug and so on may depend on these dates.

17. Blocks/ depends On: It is possible that a particular bug has a dependency on another bug. If the reported bug blocks another bug from being tested, this can be mentioned in the bug report. Similarly, if the closure of another bug is a pre-condition to observe the reported problem, you can mention the Id of that bug.

The bug reporter should carefully review the completed bug report. Usually, it is possible to improve the content of the bug report. You may use the following guidelines to review your bug report:

1. Do not be in a hurry to submit the bug report. You can come up with several ideas to improve the bug report if you give yourself a little time to think about the bug report.

2. Ensure that the bug report is within the scope of the current test. Another thing to ensure is that the bug report is against a completed feature/ function/ module of the system under test.

3. Ensure that the bug report is as compact as feasible.

4. The bug report should not directly or indirectly point to any person or any role in the project or organization. The bug report should only provide factual data about the observed problem.

5. Sometimes a few bugs are discovered in a single area of the system under test. It may be feasible to group some such bugs in a single bug report. This is another benefit of taking some between writing the bug reports and submitting them.

6. Ensure that the bug report is not a duplicate of any other bug report. It makes sense to search the defect tracking system using different searches to find any existing duplicates.

The self-review and revision is an iterative process. You should iterate this process until you are very satisfied with the bug report. Now, it is time to request a colleague or a senior to review the bug report and point out problems or improvement areas. You should then address their review comments in your bug report. Note that sometimes it makes sense to bunch a group of bug reports and ask for their collective review.

I hope that my reader is now conversant with the proper use of different fields of the bug report and the guidelines for effective reviews and revisions of bug reports. Happy bug reporting!

Learn more by watching the video, How to report bugs effectively?

10 comments:

  1. As blogs go, that's quite a lot to take in at one time.

    I've also commented on your posting at LinkedIn QAGuild forum.

    You open well. You sow the seed that this is based on experienced input, and will have hints and tips. But then you conclude by sounding pompus with "... the proper use of different fields ...".

    "Proper" and "hints & tips" don't seem to sit well in the same article.

    I think that it all got away from you a little here. Don't get me wrong, there is some good stuff here - but as hints and tips.

    ReplyDelete
  2. Inder, also check what bugzilla help says about bug reports, its what you have said and somethings more. its a nice read.

    I have a different opinion on point 2 and 5 of your guidelines. As for point 2, i would like any testing engineer to report anything that he finds unacceptable as a bug. To a some extent i am OK with invalid bugs, than not reported bugs.

    For point 5 coupling issues create a problem, as a page might have 6 issues, and u decide to fix 3, 2 are invalid and 1 is deferred, then that bug report management becomes messy..

    ReplyDelete
  3. Thanks for the post but for me it seemed to be too 'general', kind of guide for housewives... I suppose every person involved into testing is quite aware of all this. So may be it would be helpful to hear about some specific examples or kind of that.

    ReplyDelete
  4. @Ankur,

    Thanks for pointing out Bugzilla help. Bugzilla – Bug Writing Guidelines are available at the URL, http://landfill.bugzilla.org/bugzilla-tip/page.cgi?id=bug-writing.html .

    Your take on guidelines no. 2 and 5 is also okay. Your approach reduces the risk of defects being lost by not having been reported or being reported along with other similar defects. Please remember that my post is about tips on bug reporting. You should use your own professional judgement to determine which of these tips would work for you, your team, your developers, your project, your organization and your client.

    Thanks for your inputs!
    Inder P Singh

    ReplyDelete
  5. @Mark,

    You have given me good feedback about the conclusion part of my post. Additionally, I feel that the conclusion is a bit too short.

    Just like you, several readers have given me good feedback about this post. I am planning to re-write the post and hope that I would be able to address your feedback in the re-write.

    Thanks again and have a good day!
    Inder P Singh

    ReplyDelete
  6. 2 remarks:

    1 - Title - which is a very importent part of any report is missing from your article - especialy in large and busy forums of bug discussion - if the title does not include the essence - wrong decision will be taken!

    "Ensure that the bug report is within the scope of the current test" - I could not understand that - a connection to test is nice, but if you find a defect, it's your responsibility to open a defect, or make sure someone else do so. You do have to make sure that the bug is in the scope of the project's requirments, but test scope is not a factor (except for reference use)here when submitting bug.

    ReplyDelete
  7. Dear @Anonymous,

    1. The "Title" field is not missing from the article :) It is just called "Summary" in it.

    2. A bug report may be rejected (wasting everyone's time) if it is not within the agreed test scope.

    Thank you for your comment!

    ReplyDelete
  8. Ashutosh ChandraMay 10, 2010 at 2:51 AM

    Hi Inder,

    Dont know if it makes sense or not in bug reports but is often asked by my collegues that we should also provide information about condition that has worked specially when we are doing a stress, load testing or boundry testing
    e.g. When i am running a traffic at 1 GBps rate i am seeing the system crashes and never recovers

    The above defect is too generic and do not give good indicator to the management whether to put resources to fix the issue and burning the time and money

    If the tester also provide information that till what condition the system works fine then it helps the management to priortise the defect easily

    Hope this is making sense

    Thanks for useful blogs

    Regards,
    Ashutosh

    ReplyDelete
  9. Sure, Ashutosh. It commonly takes substantial time and resources to set up, run and analyze a stress/ load test. Even after setting up and running a stress/ load test once, no obvious performance issues may be discovered. For example, a throughput of 1 GBps may be acceptable with the given application and infrastructure.
    When performance issues are indeed discovered, a simple bug report may not suffice (as you have rightly mentioned). The performance issue needs to be supported by the following information. Only then, can the management and the team take an informed decision whether to go ahead and expend time and resources on the issue.
    1. Test approach used
    2. Test environment configuration (application build used, scenarios and load used, load test tool used, run-time settings used, test run time, application technical infrastructure e.g. servers and network used and so on)
    3. Analysis of the results (with supporting data and statistical interpretations)
    A single (series of) test(s) may reveal one or more performance issues. Sometimes, the team doing the performance tests also undertakes performance engineering. In such a case, there might be recommendations associated with the performance issue.

    You are right. A simple bug report may not communicate all the necessary information about a performance issue. The above post is more biased towards functional bugs and not performance issues. Thanks for leaving your comment.

    ReplyDelete
  10. Hi,
    is it just me or is it not possible to permanently disable the tracking of own site visits on blogger.com?
    There is an option in statistics but there is no "apply" button so in my case (both with Safari and Chrome) some times the own visits of my travel blog are still shown in statistics, some are not. And always after some days, the checkbox is again not selected.
    thanks

    ReplyDelete