Sunday, May 15, 2011

Software testing myths

Myths abound in the software development industry. Software testing is no exception. People tend to hear such statements and pass it on to others minus the full context. Not only that, they even tend to use such myths when making own decisions, especially when in hurry.

1. A ratio of 1 tester to 5 developers is enough to get good test coverage.
This myth may be true in long-term new development projects with multiple stages of testing (e.g. SIT, alpha, beta and UAT). This ratio may fall short in:
a. Maintenance projects with highly inter-dependent components, requiring testing of all impacted components
b. Projects requiring huge effort to create an independent test environment
c. Projects requiring extensive regression tests (due to, say, business impact, contracts or regulations)

2. Anyone can test software.
Anyone can test software if they know what to do. A professional tester is well-versed in the project requirements, software testing concepts, test design and test execution techniques and is also an assertive communicator.

3. Developers can test software better than testers. After all, they are the ones who developed it.
While developers know the code they develop intimately, they may not be the best people to test it (or test code written by other developers), especially at the integration level or system level. Testers execute their tests in a clean test environment with a fresh mindset, doing both positive and negative testing. Developer testing complements (and does not substitute) testers' testing.

4. Test cases are only derived from requirements.
Requirements are only one input to write test cases. A professional tester uses many other inputs like design documents, standards, checklists, prior test cases, past bugs reported by the customers and prior versions of the application/ similar applications.

5. Testers need only execute a set of test cases.
A static test case suite is unlikely to lead to the discovery of new bugs in the application. Before execution, the test suite should be updated by removing the useless or redundant test cases, completing or correcting the existing test cases and adding new test cases to increase the likelihood of discovering fresh and interesting bugs in the application.

6. Tests already executed need not be repeated in the release.
It depends. If the new builds do not impact the application areas covered by the tests already executed. Much more likely, some tests need to be re-executed to re-gain confidence in those areas.

7. Testers test every permutation and combination of inputs a user can provide to the application.
This is not possible except in very trivial cases. Even testing all possible inputs of a form with a few text, date and numeric input controls within the given time may well be impossible. What testers actually do is to design the minimum number of input sets that tests each input control. Specific inputs to each control are limited by using testing concepts like equivalence partitioning.

8. Testers can test equally well (if not better) when the time is short.
Nobody can perform a thorough job if rushed. Testers are no exception. If the available time is reduced too much, a tester prioritizes the tests to execute and keeps executing the higher priority tests until time runs out.

Image: Idea go /


  1. Simply wish to say your article is as amazing. The clearness in your post is just spectacular and i can assume you’re an expert on this subject. Thanks for sharing.

  2. Hi, I am an undergraduate student working on a paper on Regression Testing. Can you give me some real-life examples where enhancement of software led to regression faults?Like, the cases where old code failed to run properly during the regression testing. Can you give me links to such case studies.? Thanks in advance.

  3. Your best bet would be to browse papers presented on regression testing at conferences (IEEE, Quality and Testing). I doubt if you would get specific bug details from other case studies. Though you may get aggregate data and/ or interpretations in these case studies.
    Thank you for your comment and good luck.

  4. Thank you for the info. It sounds pretty user friendly. I guess I’ll pick one up for fun. thank u

    Software Product Development