Like other professionals, software testers continuously attempt to add value to their work. Provided they are able to take time from their tests, they attempt the following in order to excel in their projects:
1. Define and tune the testing methodologies in use (process improvement)
2. Find out and use any automation tools to provide more/ regular test coverage or speed up their existing tests (new or more test automation)
3. Report test results to the stakeholders in a more useful way (better communication)
One improvement area that is overlooked by many testers is their business or functional knowledge related to their project. In this short post, I would like to draw the attention of my reader to this point.
Professionals from many backgrounds work in the software testing field. Some professionals have vast experience in the testing field. They are aware of many ways to test, they might have good people skills and reporting skills and they might be aware of many resources and tools that they could leverage in their tests. Some other professionals are ex-developers and they bring their technical acumen to testing. They usually have rich experience in at least one development environment, they are quick to conceptualize the implementation of the software they test and they can learn any new test automation tools very quickly. There is a third category of professionals who work in the software testing field. They come from the business background. They have been in the industry for a long time and they are keenly aware of the general requirements of the business and the end-users. Of course, these professionals from the business background might not know a great deal about testing methodologies or test automation tools but they know the business.
If you work in the software-testing field and you do not belong to the third category of professionals, you should know that building up your business knowledge would put you ahead of many colleagues and fellow testers who do not choose to build that knowledge actively.
Let us look at the various ways in which you can increase your business knowledge related to your project. Many software testers who have not worked in prior projects related to an industry e.g. investment banking, retail and so on learn about the project from the available documentation and the knowledge given to them by their project team members. The documentation may exist in the form of a requirements specifications, design specifications or user stories. The project team members may provide them knowledge in meetings or casual interactions. However, the initial learning from the team and continuous learning from the feedback given by the business are not the only ways to get business knowledge. You should consider the other ways to build up your business knowledge:
1. Read the business publications (news, articles, web casts and so on).
By doing this, you would quickly become aware of the vocabulary used in the business.
2. Network and interact with professionals in the business.
This is a quick way to understand the key concepts used in the business
3. Get your hands on the documentation of similar projects done in the past and read it.
This would make you aware of the past issues and if it is a long-running project, you might become privy to information that is known to few people in your team or company.
4. If possible, work in projects related to a particular industry.
This way, you would be able to leverage your existing business knowledge.
You can safely assume that either some software testing professionals would not consider actively building up their business knowledge or even if they do so, they might not make a concerted effort in that direction. Therefore, if you make a regular effort to build up your business knowledge further, you would get the following benefits:
1. The requirements of your project would be clearer to you.
2. You would be able to focus on the key aspects of the project easily.
3. Your reports (test results and defect reports) would contain the language of the business. You would strike rapport with the business stakeholders in this way.
Strengthen your business knowledge and excel in testing.
Sunday, May 17, 2009
Sunday, March 22, 2009
Effective bug reports
Each one of us in the software testing field has seen many bug reports. Bug reports are written in a vast variety of styles. Sometimes, we happen to see the bug reports created by experts and inspire ourselves by doing so. However, a test engineer may create a bug report in any manner as seen fit. If so, can one be sure that the bug report is best suited to the circumstances? Will the bug report do the job well? In this article, I wish to make my reader aware of various aspects of a bug report. I have attempted to integrate many methods used by experts. This will enable you, my reader, to create bug reports that are best suited to the conditions at hand. The next time you have to write a bug report, you will have many tips to choose. Alternately, you may implement certain parts of some tips to create a custom strategy that suits the purpose.
The first thing that we need to do is to understand the meaning of a bug report. Put simply, a bug report is a problem statement. It describes a problem observed in the system under test. Along with the problem, a number of supporting data elements is usually present. A bug report is used to communicate important problem data between testing, development, support and management groups. It is possible that the bug report created by the reader is read and acted upon in a different country and/ or in a different organization. All the currently open bug reports collectively contribute to indicate the present quality of the system.
Next, let us look at the various data elements that can exist in a bug report. Depending on the reporting requirements or your wishes, you can use the fields available in the defect tracking system to convey the bug to the intended audience clearly.
1. Id: Each bug should have a unique identifier. If a defect tracking system is being used, this Id is usually generated automatically. If not, you can assign a unique Id to the bug report. Since managers use bug Ids to track the bugs and the bug Ids are present in past reports, you should never alter the bug Ids.
2. Summary: The summary is a very important part of the bug report. Ideally, the summary should capture the core of the problem in a few words. A summary can contain other supporting information e.g. project/ product version, build, component, platform/ environment, team name, name of the bug reporter etc. You should consider the norms of writing bug reports in your project/ organization and summarize appropriately.
3. Project/ product: The bug reporter usually files the bug report against a single project or product in the system under test.
4. Component/ Module: If the bug belongs to a particular part of the system, you should name that part of the system. On the other hand, if it were not possible to attribute the bug to a particular part of the system, you may not name the Component/ Module or may term the bug as a “system-wide” bug.
5. URL: While testing web applications, it is common to state the URL that is required to retrace the steps and observe the problem.
6. Build: The Build number is the build (major number and/ or minor number) in which you discovered the problem.
7. Type of bug: The Type of bug can be a functional problem, a data problem, a performance problem, a problem with the build/ installer or a problem with the user interface.
8. Priority: Though you can set up the priority of the bug report based on the anticipated impact of the problem on the users, development or management may set the priority value differently, if required.
9. Severity: The Severity value is set by the bug reporter based on the problem itself e.g. it can be showstopper or blocker or a high severity bug or a medium severity bug or a low severity bug.
10. OS/ Platform/ Environment: If testing was done in different environments e.g. different operating systems (Windows XP, Vista, Linux etc.) or different browsers (IE, Firefox, Safari etc.), it is useful to specify the particular environment, in case the bug is observed in that environment only. For bugs that exist in all environments, you may mention the extent of the problem in the Description field.
11. Description: This is perhaps the most important part of the bug report. It contains the basic steps required to reproduce the problem. Sometimes, it is useful to mention both the expected results as well as the actual results encountered on execution of the steps. The actual results are the problem statement. In order to obtain support to get the bug fixed, it may be useful to state the impact of the actual results on the users. You may mention any data element that does not fit in the fields available in the defect tracking system in the Description part of the bug report under appropriate heading(s).
12. Attachments: A screen capture or a short video (consider the size of the attachment) showing the problem is usually quite effective in explaining the problem to the recipients of the bug report. This file should complement the Description of the bug report. If the file shows a number of items, encircling or otherwise highlighting the problem area in the file would save the recipients time to understand the problem. In addition to screenshots/ videos, you may attach other items e.g. error log, a report, other documents to build support for bug fixing.
13. Reporter: In many defect-tracking systems, this field is automatically populated. In case it is not, you should mention your name and contact information here.
14. Assigned To: You could keep any incomplete (or “in progress”) bug report assigned to yourself. A completed bug report should be assigned to the correct person in development (a development manager/ lead or a developer). A fixed bug report should be assigned to the bug reporter so that it may be re-tested and marked as closed. In case the assignee requires information or a decision from another stakeholder e.g. management or marketing, the assignee may assign the bug report to the concerned person in the relevant group. Note that at any particular time, the bug report should stay assigned to exactly one person.
15. Cc: Often, some persons are interested in the bug reports logged against the system under test. However, they may not play any direct role in fixing the bug. Such persons can include leads and management. These persons should stay informed about any changes to the bug report.
16. Dates: A single bug report can include a number of dates. Common dates included in the bug report include Date Created, Date Assigned, Date Resolved, Date Reopened, and Date Closed. In case the date fields are not automatically maintained by the defect tracking system, care should be taken to log the correct dates. A number of testing metrics e.g. bugs opened/ closed, average age of an open bug and so on may depend on these dates.
17. Blocks/ depends On: It is possible that a particular bug has a dependency on another bug. If the reported bug blocks another bug from being tested, this can be mentioned in the bug report. Similarly, if the closure of another bug is a pre-condition to observe the reported problem, you can mention the Id of that bug.
The bug reporter should carefully review the completed bug report. Usually, it is possible to improve the content of the bug report. You may use the following guidelines to review your bug report:
1. Do not be in a hurry to submit the bug report. You can come up with several ideas to improve the bug report if you give yourself a little time to think about the bug report.
2. Ensure that the bug report is within the scope of the current test. Another thing to ensure is that the bug report is against a completed feature/ function/ module of the system under test.
3. Ensure that the bug report is as compact as feasible.
4. The bug report should not directly or indirectly point to any person or any role in the project or organization. The bug report should only provide factual data about the observed problem.
5. Sometimes a few bugs are discovered in a single area of the system under test. It may be feasible to group some such bugs in a single bug report. This is another benefit of taking some between writing the bug reports and submitting them.
6. Ensure that the bug report is not a duplicate of any other bug report. It makes sense to search the defect tracking system using different searches to find any existing duplicates.
The self-review and revision is an iterative process. You should iterate this process until you are very satisfied with the bug report. Now, it is time to request a colleague or a senior to review the bug report and point out problems or improvement areas. You should then address their review comments in your bug report. Note that sometimes it makes sense to bunch a group of bug reports and ask for their collective review.
I hope that my reader is now conversant with the proper use of different fields of the bug report and the guidelines for effective reviews and revisions of bug reports. Happy bug reporting!
--- Inder P Singh
The first thing that we need to do is to understand the meaning of a bug report. Put simply, a bug report is a problem statement. It describes a problem observed in the system under test. Along with the problem, a number of supporting data elements is usually present. A bug report is used to communicate important problem data between testing, development, support and management groups. It is possible that the bug report created by the reader is read and acted upon in a different country and/ or in a different organization. All the currently open bug reports collectively contribute to indicate the present quality of the system.
Next, let us look at the various data elements that can exist in a bug report. Depending on the reporting requirements or your wishes, you can use the fields available in the defect tracking system to convey the bug to the intended audience clearly.
1. Id: Each bug should have a unique identifier. If a defect tracking system is being used, this Id is usually generated automatically. If not, you can assign a unique Id to the bug report. Since managers use bug Ids to track the bugs and the bug Ids are present in past reports, you should never alter the bug Ids.
2. Summary: The summary is a very important part of the bug report. Ideally, the summary should capture the core of the problem in a few words. A summary can contain other supporting information e.g. project/ product version, build, component, platform/ environment, team name, name of the bug reporter etc. You should consider the norms of writing bug reports in your project/ organization and summarize appropriately.
3. Project/ product: The bug reporter usually files the bug report against a single project or product in the system under test.
4. Component/ Module: If the bug belongs to a particular part of the system, you should name that part of the system. On the other hand, if it were not possible to attribute the bug to a particular part of the system, you may not name the Component/ Module or may term the bug as a “system-wide” bug.
5. URL: While testing web applications, it is common to state the URL that is required to retrace the steps and observe the problem.
6. Build: The Build number is the build (major number and/ or minor number) in which you discovered the problem.
7. Type of bug: The Type of bug can be a functional problem, a data problem, a performance problem, a problem with the build/ installer or a problem with the user interface.
8. Priority: Though you can set up the priority of the bug report based on the anticipated impact of the problem on the users, development or management may set the priority value differently, if required.
9. Severity: The Severity value is set by the bug reporter based on the problem itself e.g. it can be showstopper or blocker or a high severity bug or a medium severity bug or a low severity bug.
10. OS/ Platform/ Environment: If testing was done in different environments e.g. different operating systems (Windows XP, Vista, Linux etc.) or different browsers (IE, Firefox, Safari etc.), it is useful to specify the particular environment, in case the bug is observed in that environment only. For bugs that exist in all environments, you may mention the extent of the problem in the Description field.
11. Description: This is perhaps the most important part of the bug report. It contains the basic steps required to reproduce the problem. Sometimes, it is useful to mention both the expected results as well as the actual results encountered on execution of the steps. The actual results are the problem statement. In order to obtain support to get the bug fixed, it may be useful to state the impact of the actual results on the users. You may mention any data element that does not fit in the fields available in the defect tracking system in the Description part of the bug report under appropriate heading(s).
12. Attachments: A screen capture or a short video (consider the size of the attachment) showing the problem is usually quite effective in explaining the problem to the recipients of the bug report. This file should complement the Description of the bug report. If the file shows a number of items, encircling or otherwise highlighting the problem area in the file would save the recipients time to understand the problem. In addition to screenshots/ videos, you may attach other items e.g. error log, a report, other documents to build support for bug fixing.
13. Reporter: In many defect-tracking systems, this field is automatically populated. In case it is not, you should mention your name and contact information here.
14. Assigned To: You could keep any incomplete (or “in progress”) bug report assigned to yourself. A completed bug report should be assigned to the correct person in development (a development manager/ lead or a developer). A fixed bug report should be assigned to the bug reporter so that it may be re-tested and marked as closed. In case the assignee requires information or a decision from another stakeholder e.g. management or marketing, the assignee may assign the bug report to the concerned person in the relevant group. Note that at any particular time, the bug report should stay assigned to exactly one person.
15. Cc: Often, some persons are interested in the bug reports logged against the system under test. However, they may not play any direct role in fixing the bug. Such persons can include leads and management. These persons should stay informed about any changes to the bug report.
16. Dates: A single bug report can include a number of dates. Common dates included in the bug report include Date Created, Date Assigned, Date Resolved, Date Reopened, and Date Closed. In case the date fields are not automatically maintained by the defect tracking system, care should be taken to log the correct dates. A number of testing metrics e.g. bugs opened/ closed, average age of an open bug and so on may depend on these dates.
17. Blocks/ depends On: It is possible that a particular bug has a dependency on another bug. If the reported bug blocks another bug from being tested, this can be mentioned in the bug report. Similarly, if the closure of another bug is a pre-condition to observe the reported problem, you can mention the Id of that bug.
The bug reporter should carefully review the completed bug report. Usually, it is possible to improve the content of the bug report. You may use the following guidelines to review your bug report:
1. Do not be in a hurry to submit the bug report. You can come up with several ideas to improve the bug report if you give yourself a little time to think about the bug report.
2. Ensure that the bug report is within the scope of the current test. Another thing to ensure is that the bug report is against a completed feature/ function/ module of the system under test.
3. Ensure that the bug report is as compact as feasible.
4. The bug report should not directly or indirectly point to any person or any role in the project or organization. The bug report should only provide factual data about the observed problem.
5. Sometimes a few bugs are discovered in a single area of the system under test. It may be feasible to group some such bugs in a single bug report. This is another benefit of taking some between writing the bug reports and submitting them.
6. Ensure that the bug report is not a duplicate of any other bug report. It makes sense to search the defect tracking system using different searches to find any existing duplicates.
The self-review and revision is an iterative process. You should iterate this process until you are very satisfied with the bug report. Now, it is time to request a colleague or a senior to review the bug report and point out problems or improvement areas. You should then address their review comments in your bug report. Note that sometimes it makes sense to bunch a group of bug reports and ask for their collective review.
I hope that my reader is now conversant with the proper use of different fields of the bug report and the guidelines for effective reviews and revisions of bug reports. Happy bug reporting!
--- Inder P Singh
Tuesday, March 17, 2009
Subscribe to:
Posts (Atom)
