Saturday, August 21, 2010

Find a Bug

Scenario: You are a competent tester who has just joined a company and is testing on the first project there. The application you are testing is a financial web application. During test execution, you make the following observations. Rate each observation on a scale of 1 to 10, 1 being definitely not a bug and 10 being certainly a bug.

1. The logo on the home page and other pages of the application is not the client's. For example, if the client's name were 1, the logo is that of another company, say 2. The logo in the previous version of the application is your client's.

2. There is a group of links on the pages of the application. When you click any link in this group, it opens a page with the "Not Found" error.

3. You log on as a new user into the application. You make a deposit of $ 1,000 into your account. When you visit the Transactions page, it shows you your transaction of $ 1,000. However, when you visit the Balance page, it shows your available balance as $ 2,000.

Should you go ahead and report these observations as bug reports pronto?
.
.
.
.
.
.
.
.
.
.

My advice is No. All your observations mean is that you should investigate further. For all you know, there might be perfectly reasonable explanations for these observations. For example,

1. Your client is getting this application developed for a partner organization. Or, their parent organization. Hence, the different logo. The change request is already under construction and you would get it soon.

2. The group of links will be supported by an external entity. During your test, you should just have checked if each link is correct. The external entity would populate these links before the test finish date and then all the links in the group would work.

3. There is indeed a requirement which states that when a new user makes the first deposit between $ 1,000 and $ 2,000, the client organization doubles the balance by making an equal deposit into the user's account. However, you are not aware of this requirement.

The objective of this post was to help you realize how using Heuristics may lead you along a wrong path. However, I did not state this objective at the beginning of the post because I did not want to bias you.

Testing is an investigative process. We should observe the application carefully, formulate theories if what we see is not in line with our mental model, gather more information from multiple sources and analyze and test our theories before pronouncing a discrepancy or a bug in the application.

P.S. The idea to write this post came after reading, ICICI Bank ATM bug … Try it!............. by Amit Jain, a member of the Software Testing Space group.