As you wish!

Once upon a time, there was a tester who had just joined a large end-user company. He had a good profile and was promptly inducted into an ongoing project to develop an application for internal use across offices of the company. Things started moving slowly for this person.

Though nobody in his team realized at the time, this tester was quite capable. He was also quite talented, hardworking and methodical in his working style. He quickly introduced himself to each of the team members (but in his hesitant way). During his initial days in the project, he made it a point to read (and re-read) all the documentation related to the project. Whenever he had questions, he made the effort to approach the relevant team member and clarify his thoughts.

Some time passed and now he was assigned some modules to test in the project. Since the tester was quite knowledgeable about the requirements by this time, he could understand the existing test cases quite well. He was also able to see the short-comings in these test cases and made refinements to his own copy of the test cases before executing them. He found defects. In fact, he found a lot of defects. This led to the problem.

The Development Lead on the project was an influential person. He was always "in the know". He was also widely recognized in the company as a competent developer and business domain expert. But somehow, he did not like so many defects being logged by our tester against "his" application and "his" team. He called the tester for a one-on-one meeting and questioned his work. Did the tester really understand the business objectives of the application? Should the tester have been spending more time validating the application? Were his defects really valid at this point in time?

Our tester did not like confrontation. So, when the Development Lead changed the following rules, our tester said "As you wish" and smiled meekly:

1. The tester would now only test the modules specifically assigned to him and not anything else.

2. If the tester thought he found a bug, he was to approach the Development Lead, take an appointment and present his bug. Only when the Dev Lead had analyzed and agreed to the bug was the tester required to report it in the bug tracking system.

3. The tester was not to approach any team members directly. If he had questions, he was to take permission from the Dev Lead and only then meet the team members.

The Development Lead did not however stop at that. During his status reporting meetings with the management, he took the time about how he thought of our tester as a somewhat loose cannon. He said that the tester needed to be monitored closely but assured the management that he had put the tester on an "improvement plan".

Life had become tougher for our tester. While testing, he could see bugs in many areas of the application but he had to keep them to himself. When he attended the meetings with the team, his mind would be full of ideas. However, many times he hesitated to share those ideas. Other times, when he would get the courage of speaking them, he was quickly interrupted by the Dev Lead, "We will talk about it later". The tester would just smile weakly and say "As you wish". Many other team members too started taking little notice of the things the tester had to say. When the tester found bugs in the modules assigned to him, he would make careful notes and take them to the Dev Lead for discussion. The Dev Lead rejected most of the bugs he thought of as "not realistic enough", "too early to fix" or "doubtful". The result was that he allowed the tester to report the most obvious bugs only. But our tester kept on plodding without regrets.

Time passed. A number of releases were made to downstream teams. Finally, the application was deployed to production. The Dev Lead was also now the in-charge of supporting the application in production. In the first few months, internal customers reported several issues in the application. These were promptly fixed. Six months after the go-live date, the Lead happened to think about the issues reported in the go-live period. He listed the issues and started analyzing them. Just out of curiosity, he picked up our testers submitted notes from the previous year to see whether he had put forward any one of the real issues. He was amazed. All save one issue reported in production against the tested modules was discussed by the tester, while the application was still in development. Now, the Lead was very ashamed of how he had treated the capable tester.

He promptly called the tester for a meeting. He hurriedly explained his pre-occupations at the time and how he now realized the good work done by the tester. The tester was now free to discuss his ideas with others and report bugs as he saw fit. Then he asked the tester if this was okay? All our tester had to say was "As you wish", again with a meek smile.

Let us treat each team member with respect. Who knows how much a team member may be able to contribute to our success?

Equivalence Partitioning - Why you should NOT use it AS IS in black box testing?

Equivalence Partitioning is an interesting test design technique. You can learn it from my video, Equivalence Partitioning and Boundary Value Analysis tutorial.

Put simply, EP involves dividing the input test data into partitions, valid and invalid. It promises to reduce the number of input test data values that must be used by suggesting that only one test data value be used from each partition to determine the behavior of the application for each partition.

Let us consider an example of the input, day of the week. The valid partition is 0..6 (or Sunday to Saturday). marked in green. There would be two invalid partitions, <0 and >6, which are marked in red.

-------------------------------------------------------------------------------------------

...  -3   -2   -1      0    1    2    3    4    5    6    7    8    9  ...

-------------------------------------------------------------------------------------------

Therefore, you decide to execute your test case with three distinct values, say -3, 3 and 9 (a value each from the first invalid partition, the valid partition and the second invalid partition). Correct?

Wrong decision. The technique assumes that the application treats any value within a particular partition identically. Well, may be NOT. Unless you are privy to the source code and can confirm that indeed each value in each partition is handled exactly the same.

Partitioning is fine. I disagree with the "Equivalence" part of "Equivalence Partitioning" term. I will mention why. Let us say that the programmer wrote separate "Case" or "Switch" statement(s) for each value from 0 to 6. If you only ever execute your test case for the input value 3, you would never test the application's code for the other values in the valid partition (0,1,2,4,5 and 6).

At this point, I can anticipate questions. How about the invalid partition? Should you execute your test case with multiple values in the invalid partition? How far back should you go? Obviously, you cannot use each value in a large partition. However, you may consider the following:

a. Use at least one or two values in the beginning of the partition e.g. -1 and -2.

b. Use at least another value deeper in the partition e.g. -100.

c. Use a value outside the range of the data type (integer in this case) e.g. -12345678

In short, when you have to test without looking at the source code, test using each input value in a small partition. And, test using spread out sample values in a large partition.

Future of software testing

We don't know the future. Nobody does. However, I have been involved with software testing since 1998 and I have seen changes taking place. It used to be simple earlier. We received the requirements and designed simple test cases based on these requirements. Then we received the latest build of the application, execute our test cases and report discrepancies in bug reports. We did all of this as carefully and as fast as we could. As I experienced software testing first hand, I began to learn it. There were multiple approaches available to perform testing tasks. Each approach had its own pros and cons. There were multiple types of testing that we could do. Each of my team members had a different set of skills, strengths and weaknesses. It was not simple anymore. 3 or 4 years ago, I started following the online software testing communities. I read a lot of material and comments on hundreds of topics in software testing. I saw thought leaders in software testing repeatedly pointing us test practitioners to the basics. Over time, I have begun to consider software testing as quite complex and challenging. The good news is that software testing still has a long way to go before it truly matures.

Here is the list of my predictions. These are not revolutionary changes that are going to catch you all of a sudden. In fact, you can see some of these changes today. But, if you are aware that these changes could speed up or intensify in the coming future, then you have a better chance to prepare yourself to take advantage of them.

Prediction #1. Companies will demand more value for the testing resources they put in.

The recent downturn has shaken everyone. We have been forced to become savvier with our investments. The same is true for companies. Companies will demand a better testing service from their resources (in-house testing team or a vendor providing testing services). The companies will now demand:

a. Faster turnaround time

b. Greater coverage of specified and implied requirements

c. Testing in more perspectives (functional, performance, security, usability and so on)

d. Increased collaboration with all other teams involved in sales/ product development, software development, deployment and support

e. Lower costs (of test infrastructure (test environment), test tools and testing personnel)

f. More transparency of the test process

Prediction #2. Software testing will become more complex.

Keeping in mind the increased expectations of clients, the increasing complexity of applications and the increasing knowledge of test practitioners, software testing will become more complex. In future, testers will need to find answers to the following questions among others:

a. What are the most important business objectives of the application that I am testing?

b. What technologies does my application employ? How do I test each of those?

c. What test infrastructure will I need to test my application? How can I set those up with the least cost (of setting it up, using it, maintaining it and tearing it down)?

d. What tests would provide the best value against the cost of creating them?

e. How is my application integrated with other systems? How do I test various aspects of each integration?

f. What is the best test methodology that I can use?

f. Which of my communication provides the best value to other stakeholders?

g. How do I utilize my natural strengths in testing? How do I circumvent my natural weaknesses?

Prediction # 3. Crowd-sourcing will continue to become popular.

uTest is becoming more popular by the day. Today, uTest has more than 20,000 testers and a client list that includes Google and Microsoft. The clients of crowd-sourcing companies can buy just the testing services they need when they need them and even select the individual testers for the test. No wonder, many companies consider crowd-sourced testing services as viable alternatives to large in-house testing teams or inflexible testing services vendors.

Prediction # 4. In order to get hired and stay hired, testers will need to distinguish themselves from the crowd.

Today, there are masses of software testers. Their profiles and resumes look similar. If I were going to hire someone for my team, I would not like to just go for someone with the basic knowledge and skills. I would like to get the details. And, I would probably like to interview someone with substantial achievements. Someone who has "walked the extra mile". Someone who has achieved more than their counterparts at the same level. Be it extra-ordinary knowledge, uncommon or advanced skills or a solid recognition from testing experts.

Prediction # 5. Social skills and working style will become important.

Other than software testing knowledge and skills, testers will be required to be socially adept. They will be required not only to plan and test well, but also communicate well. They will be required to establish themselves as part of a team, support the team, speak up when required and influence others when required. In future, just testing won't do for the testers. They will also be required to collaborate effectively and strive to maintain long-term relationships with their extended teams. Further, testers will be required to show align themselves to the (stable or changing) business objectives and the team.

What do you feel? Do you think that you are ready for these changes? Are you going to take advantage of them? What other changes do you see on the horizon?