No Time to Test

There was a thought-provoking discussion in the Software Testing Space LinkedIn group this month. The problem posed by Amit (many thanks for raising an important practical problem) was thus.

Problem One is the only QA engineer for a large team of developers. The rate at which the developers build new features is far greater than the rate at which the QA engineer can test them. How would one ensure high quality is maintained in the application?

Here are the solutions to this common problem that were proposed by the expert group members:

1. Involve the developers in the team for some testing.
2. At first, test the high business priority features and then the high-risk features. Perform a regression test whenever time permits.
3. Prioritize all features to be tested. Test the new features and important bug fixes first.
4. Raise the problem to the team highlighting the limited test coverage due to lack of time. Make sure that the team understands and accepts the risk.
5. Use test automation to automate sanity test and regression test cases. Build the test automation framework so that developers can easily build automated test scripts.
6. Merge test cases or write workflow-based test cases that allow more coverage and take less effort to write and execute.
7. Use Requirement and Risk based testing approach by defining the testing scope based on priority, impact and timelines.

Overall, the members agreed on the best solutions as no. 4 and no. 1 and 5. Michael neatly summarized the solution. Many thanks, Michael.

Solution Ensure that the team is aware of the limitations of time and resources for testing new features and bug fixes and regression testing. The team needs to understand the risk due to these limitations. Always do your best. Ask and accept help from any team member with your testing tasks.

JMeter Web Performance Testing Training Course

I have launched this course in web performance testing using the leading open source free tool, Apache JMeter. As you may know, I have a number of years experience in delivering performance testing and load testing projects successfully. I started this course on repeatedly finding even experienced software testing professionals having many gaps in their knowledge of performance testing and load testing. This course has 13+ hours of online training videos with lots of example performance test plans. The topics taught in this course include performance testing concepts, JMeter installation and using the UI effectively, using Proxy Server, building requests, Thread groups, Logic controllers, Samplers, Listeners and statistical results, data parametrization, test script and test plan modeling, server technologies, profilers and many more.  In order to get started, please see my demo videos

JMeter short tutorial

JMeter detailed tutorial

You can see more details of these training videos at JMeter Web Performance Testing Training Videos.

Performance Test Reports - JMeter Listeners Tutorial

I run a training course on Apache JMeter called Web Performance Testing with JMeter. One of the important concepts in JMeter is Listeners and that I want to share with you. Reports are called Listeners in JMeter vocabulary. Listeners are used to collect the performance test results and display it to the performance tester. Now, let us learn about Listeners in JMeter and how to use them. You can see listeners working in my short JMeter video.

Listeners are used to collect and display performance test results. But there are many types of Listeners. One of my favorites is View Results in Table. It is a simple Listener that shows data about each response. Here is what it looks like.

In the above Listener, the Sample Time (ms) indicates the Response Time, the Status indicates if the request was successful (green means yes, orange means no), the Bytes indicate the size of the response and the Latency indicates the Latency Time. Of course, you can copy the test results to Microsoft Excel and format them further. But, there is an easier option. You can have JMeter save the test results to a CSV file or XML file. Further, you can even specify which fields you want to save by clicking the Configure button. JMeter can also summarize the test results automatically for you. For this, you can use Summary Report. Here is what it looks like. It summarizes the results for each request. 

JMeter has Listeners that automatically show the test results graphically. There are several like Spline Visualizer and Distribution Graph but my favorite is Graph Results. Here is what it looks like. It shows the Average and Median Response Time and the Throughput.

Some other points to note are:

1. All Listeners have access to the same test results. Only their display is different.

2. As with other test elements, Listeners work according to their scope. This means that if you put a Listener under the Test Plan, it will capture and show all the test results. If you put the Listener a request, it will capture and show test results of that request only.

3. In a particular scope, Listeners are last in execution order because they need all the results as input.

4. Listeners use substantial CPU and RAM resources on the computer. In order to avoid the Listeners becoming a bottleneck, you should decide the best Listener for your purpose and put just that Listener in your Test Plan.

Overall, JMeter Listeners are basic but configurable and very simple to use. You can save effort on test results reporting by using these available Listeners.