Why your bugs are rejected (20+ reasons why developers can reject your bug reports)?

Scenario: You test your application. You discover some bugs. You are happy. You craft your bug reports and log them in the bug tracking system. The next day, you find that a number of the bugs submitted by you have been rejected by the developers. What may be the reasons? First, view my Bug Reporting tutorial. Then read below.

You may face bug rejections even when you are "seemingly" careful in your testing and bug reporting activities. Of course, you want your tests to be taken seriously. Do you know the various reasons why your bugs may be rejected? Reflect on the reasons given below and consider them when reporting bugs.

Bug rejection reasons related to requirements
1. There is no documented requirement that mentions the result expected in your bug report.
2. You are not aware of all the (minor) details (and caveats) of a requirement.
3. The requirement that you thought was not implemented is indeed implemented in another way. But you are not aware of this decision.
4. You are not aware about the changes to a particular requirement.
5. Your bug report relates to some work that has not started yet. Or, the work has started but has not been submitted for test yet.
6. Your bug report refers to some dummy test data in the application. This bug would not exist when the real/ realistic data is set up in the application.
7. The bug is in an area of the application not within the scope of development e.g. external systems, external content etc.
8. It has been like this since the beginning or for a long time OR the users have always accepted this result.

Bug rejection reasons related to test environment
9. The settings (e.g. language, region and fonts) on your test system are incorrect.
10. The application settings (e.g. settings in the administration section) used by you are incorrect.
11. Some required files are missing on your test system.
12. The required programs are missing on your test system. Or, you have the incorrect versions of the required programs.
13. The application does not support the platform (e.g. operating system and its versions or browser and its versions) tested by you.
14. You have tested against an old build. The bug is not present in the current build submitted for test.

Bug rejection reasons related to test data
15. The test data used by you is unrealistic.

Bug rejection reasons related to the test steps/ test case
16. The steps described in your bug report are unrealistic.

Bug rejection reasons related to the bug report itself
17. The Title (Summary) and Description of the bug report are mismatched.
18. It is not possible to understand your bug report (due to incomplete information provided or other reasons).
19. It is a duplicate bug report. For example, it is a complete duplicate of another bug report OR it describes the same problem that occurs in another area of the application as given in a valid bug report OR it describes a problem that is a part of the problem described in another valid bug report)

Bug rejection reasons related to management decisions
20. A long time after you submitted the bug, it is decided that it is not feasible to put in the effort required to fix it.
21. Due to strategic changes in the plan, all bug reports related to specific application functionality are rejected. Your bug report is one of those that are rejected.

Other reasons
22. The bug report describes a problem that is due to another bug. When this other bug would be fixed, the problem mentioned in your bug report would not exist.
23. The bug is too minor.

Once your bug is rejected, you would do well to remain calm. Read your bug report carefully and try to reproduce the bug. Then, analyze the reason for the bug rejection, establish whether it is a valid bug or not and take appropriate action. If it really an invalid bug, treat this event as a learning experience. If you think that it is a valid bug, you should confidently take further action in support of your bug report. Either way, you would become more mature as a tester. However, remain motivated and enthusiastic even when you mature. Motivation and enthusiasm play a big part in your testing success.

 

Have you ever had a bug rejected? What was the reason? Share it in your comment.

Software Test Estimation: How to estimate testing efforts (6 approaches to get test effort estimate)?

Test effort estimation is a skill required of a Test Lead or a Test Manager. However, test estimation is not a skill that one can learn quickly. It requires understanding of several key concepts and practice. In this post, I will explain what test effort estimation is, point you to your existing knowledge of estimation and provide you the key concepts that you can use in your estimation. First, view my video on Test Estimation techniques with formula example and Questions and Answers. Then read below.

First of all, we should understand what we mean by software test effort estimation. Test effort estimation is answering two basic questions about testing:
I. What will be done?
II. How much effort would it take?

There are other questions e.g. I. Who will do what? II. When will they do it? III. How will they do it? but these questions are not related to effort estimation but to planning and scheduling.

Even if you have not estimated test effort before (having relied on the effort estimates given by the client or your project manager), keep in mind that you do effort estimation on a regular basis. Let me explain. Do you recognize the following situations?

1. You are appearing in an examination. The duration of the examination is 3 hours and you have to answer n questions. You average the time available for answering one question while leaving out certain time for revision at the end. You look at the questions. Some questions are easy for you but some are not. You reserve less time than average for answering the simple questions and more time than average for the difficult ones.

2. You have to attend a job interview. The interview is at 10 a.m. You estimate the time it would take you to reach the interview venue, say 1 hour. You add some time e.g. 30 minutes for delays like traffic snarls. You estimate some time, say 30 minutes for collecting your documents and some time, say 30 minutes for dressing up. This means that you would need to wake up no later than 7:30 a.m. that morning to reach your interview venue in time.

3. It is the beginning of another day at work. Your manager has given you 20 test cases to execute today. In addition, you need to complete the annual self-appraisal form. You estimate that it would take you 1 hour to complete your appraisal form. Out of 8 hours of your work day, you have 7 hours remaining. You reckon that you need to execute a test case every 21 minutes (7 hours X 60 minutes / 20 test cases).

If the above situations look common to you, it means that you already do effort estimation even if you do not consciously recognize it as such.

How to estimate testing efforts?

Next, let us see the factors that you need to consider before you do test effort estimation:

a. Size of the system
It would take longer to test a larger system. In some projects, it is possible to know about the size of the system in terms of Function Points, Use Case Points or Lines of Code. You should take the size of the system into account when estimating the test effort.

b. Types of testing required
Sometimes, it is important to perform multiple types of testing on the system. For example, other than functional testing, it may be necessary to perform load testing, installation testing, help files testing and so on. You should create the effort estimates for each type of testing separately.

c. Scripted or exploratory testing
It may be feasible to only execute test cases or do exploratory testing or do both. If you intend to do scripted testing and do not have test cases available, you should estimate the time it would take to create the test cases and maintain them. Scripted testing requires test data to be created. If the test data is not available, you should estimate the effort it would take to create and maintain test data.

d. "Non-testing" activities
Apart from creating and executing tests, there are other activities that a tester performs. Examples include creating test logs/ reports, logging defects and entering time in the project management tool.

e. Test cycles
By a test cycle, I mean a complete round of testing (build verification testing followed by attempted execution of all test cases followed by all defects logged in the defect tracking system). In practice, one test cycle is not sufficient. You should estimate the number of test cycles it would take to promote the system to the client or production.

Now, let us understand the various approaches that you can use for test effort estimation. You may choose any of these approaches for your estimation. However, in my opinion, a combination of multiple approaches works best (by best, I mean that the effort estimates are close to the real actual efforts). In any case, you should be aware about the following approaches:

1. Use historical data from other projects
This approach is useful when you have effort data available from earlier projects which are very similar to the current project. For example, this approach is useful in the case of long-running projects where the test effort data from previous releases is readily available.

2. Your organization's approach
Your organization may have their custom approach to estimate test effort in projects.

3. Delphi method
This is useful when you have a number of experts knowledgeable in the testing to be done. The experts estimate separately and then their estimates are consolidated.

4. Use your own expert judgment
This approach is useful to arrive at a rough test effort estimate quickly.

5. Software size based approach
If the size of the system is available and the formula to convert software size to test effort is available, this approach may be used.

6. Activities based approach
This approach is useful if you can list the activities required. This approach may be used Top-Down (listing the high level activities and breaking them down to lower level activities) or Bottom-Up (listing the individual activities and combining them to higher level activities). Using this approach in the Top-Down manner is better since you can control the level of detail in your effort estimate. Remember to consider activities for each type of testing, any test cases or test data need to be created, the "non-testing" activities and the multiple test cycles.

As I mentioned before, you may choose any approach to do your test effort estimation. However, using at least two approaches is better. This way, you can compare the test effort estimates and address any obvious problems. Whatever hybrid approach you choose, you should document the assumptions made by you in the estimate.

Once you have arrived at the test effort estimate for your project and have convinced the stakeholders that it is a reasonable estimate, it does not stop there. You should track the actual progress in your project constantly to see if it is in line with your test effort estimate. You may find that some of your assumptions were not correct. You should revise your assumptions and your approach in line with your observations.

Continue to use your refined test effort estimation approach across test cycles and releases. In time, you should have a good estimation approach available with you.

Refer the Q and A related to software test effort estimation. These guide you during your test effort estimation and in your discussions with project stakeholders. In addition, these make for great interview questions.

See also:

How to create a powerful Test Strategy

Roles and Responsibilities of a Software Test Lead

Responsibilities of a Software Test Engineer

How to do real database testing (10 tips to perform serious database tests)?

Many (but not all) applications under test use one or more databases. The purposes of using a database include long-term storage of data in an accessible and organized form. Many people have only a vague idea about database testing. If you are serious about learning database testing, view the videos, Database Testing and SQL Tutorial for Beginners. Then read on...

Firstly, we need to understand what is database testing? As you would know, a database has two main parts - the data structures (the schema) that store the data AND the data itself. Let us discuss them one by one.

Database testing

The data is stored in the database in tables. However, tables may not be the only objects in the database. A database may have other objects like views, stored procedures and functions. These other objects help the users access the data in required forms. The data itself is stored in the tables. Database testing involves finding out the answers to the following questions:

Questions related to database structure
1. Is the data organized well logically?
2. Does the database perform well?
3. Do the database objects like views, triggers, stored procedures, functions and jobs work correctly?
4. Does the database implement constraints to allow only correct data to be stored in it?
5. Is the data secure from unauthorized access?

Questions related to data
1. Is the data complete?
2. Is all data factually correct i.e. in sync with its source, for example the data entered by a user via the application UI?
3. Is there any unnecessary data present?

Now that we understand database testing, it is important to know about the 5 common challenges seen before or during database testing:

1. Large scope of testing
It is important to identify the test items in database testing. Otherwise, you may not have a clear understanding of what you would test and what you would not test. You could run out of time much before finishing the database test.
Once you have the list of test items, you should estimate the effort required to design the tests and execute the tests for each test item. Depending on their design and data size, some database tests may take a long time to execute. Look at the test estimates in light of the available time. If you do not have enough time, you should select only the important test items for your database test.

2. Incorrect/ scaled-down test databases
You may be given a copy of the development database to test. This database may only have little data (the data required to run the application and some sample data to show in the application UI). Testing the development or test or staging databases may not be sufficient. You should also be testing a copy of the production database.

3. Changes in database schema and data
This is a particularly nasty challenge. You may find that after you design a test (or even after you execute a test), the database structure (the schema) has been changed. This means that you should be aware of the changes made to the database during testing. Once the database structure changes, you should analyze the impact of the changes and modify any impacted tests.
Further, if your test database is being used by other users, you would not be sure about your test results. Therefore, you should ensure that the test database is used for testing purpose only.
You may also see this problem if you run multiple tests at the same time. You should run one test at a time at least for the performance tests. You do not want your database performing multiple tasks and under-reporting performance.

4. Messy testing
Database testing may get complex. You do not want to be executing tests partially or repeating tests unnecessarily. You should create a test plan and proceed accordingly while carefully noting your progress.

5. Lack of skills
The lack of the required skills may really slow things down. In order to perform database testing effectively, you should be comfortable with SQL queries and the required database management tools.

Next, let us discuss the approach for database testing. You should keep the scope of your test as well as the challenges in mind while designing your particular test design and test execution approach. Note the following 10 tips:

1. List all database-specific requirements. You should gather the requirements from all sources, particularly technical requirements. It is quite possible that some requirements are at a high level. Break-down those requirements into the small testable requirements.

2. Create test scenarios for each requirement as suggested below.

3. In order to check the logical database design, ensure that each entity in the application e.g. actors, system configuration are represented in the database. An application entity may be represented in one or tables in the database. The database should contain only those tables that are required to represent the application entities and no more.

4. In order to check the database performance, you may focus on its throughput and response times. For example, if the database is supposed to insert 1000 customer records per minute, you may design a query that inserts 1000 customer records and print/ store the time taken to do so. If the database is supposed to execute a stored procedure in under 5 seconds, you may design a query to execute the stored procedure with sample test data multiple times and note each time.

5. If you wish to test the database objects e.g. stored procedures, you should remember that a stored procedure may be thought of as a simple program that (optionally) accepts certain input(s) and produces some output. You should design test data to exercise the stored procedure in interesting ways and predict the output of the stored procedure for every test data set.

6. In order to check database constraints, you should design invalid test data sets and then try to insert/ update them in the database. An example of an invalid data set is an order for a customer that does not exist. Another example is a customer test data set with an invalid ZIP code.

7. In order to check the database security, you should design tests that mimic unauthorized access. For example, log in to the database as a user with restricted access and check if you can view/ modify/ delete restricted database objects or view or view and update restricted data. It is important to backup your database before executing any database security tests. Otherwise, you may render your database unusable.
You should also check to see that any confidential data in the database e.g. credit card numbers is either encrypted or obfuscated (masked).

8. In order to test data integrity, you should design valid test data sets for each application entity. Insert/ update a valid test data set (for example, a customer) and check that the data has been stored in the correct table(s) in correct columns. Each data in the test data set should have been inserted/ updated in the database. Further, the test data set should be inserted only once and there should not be any other change in the other data.

9. Since your test design would require creating SQL queries, try to keep your queries as simple as possible to prevent defects in them. It is a good idea for someone other than the author to review the queries. You should also dynamically test each query. One way to test your query is to modify it so that it just shows the resultset and does not perform the actual operation e.g. insert, delete. Another way to test your query is to run it for a couple of iteration s and verify the results.

10. If you are going to have a large number of tests, you should pay special attention to organizing them. You should also consider at least partial automation of frequently run tests.

Now you should know what database testing is all about, the problems that you are likely to face while doing database testing and how to design a good database test approach for the scope decided by you.

See also:

What is Database

Database Migration Testing

Sample database test plan

Database testing example

Another database testing example

SQL test

SQL injection