Why bad software testing happens?

It is possible for software testing to go wrong in a project. Look out for these red flags in your project and make corrections immediately to avoid bad testing.

1. Customer inputs ignored

2. Poor knowledge of the application under test

3. Application technology ignored

4. Incorrect test strategy

5. Old test process followed mechanically

6. Unorganized testing

7. Poor observation and analyses

8. Insufficient communication

9. Lack of perseverance

10. No self-critique/ correction in the team

Automated test script review checklist

Reviews are a great way to find common problems with a number of test automation artifacts, such as automated test scripts like the ones that I wrote in my Selenium Python tutorials. By an automated test script, I mean script code implementing a test case's test steps and expected results e.g. the second script in my Selenium Python Tutorial 2. The automated test script may be written in any programming language or scripting language. It may be generated by a functional test automation tool or written by hand.

Here is a checklist that you can use to review your or others automated test scripts. You should customize the questions according to your specific needs. For practice, review the automated test script to answer a quiz automatically. Post your review comments below.

Inputs

1. Does the script implement each step of the test case?

2. Does the script implement each step in the correct order?

3. Does the script implement each validation given in the test case?

4. Does the script have each validation after the correct step?

Execution

5. Does the script logic call the correct functions? Are these functions called with the correct arguments?

6. Does the script take application input values from the correct input test data file(s)?

7. Does the script execute user operations against the correct objects?

8. Does the script give the application sufficient time to transition to the correct test state e.g. the correct screen?

9. Does the script validate application output values against the correct output test data file(s)?

10. Does the script validate the state of the correct objects?

11. Does the script load only the necessary objects in memory?

12. Is the script free from unnecessary delays?

13. Does the script handle application errors as designed?

Miscellaneous

14. Does the script have a unique identifier?

15. Is the script free of any magic numbers or magic strings?

16. Is the script commented at each appropriate place in it? Is each comment correct?

17. Is the script free from unnecessary or extra code?

Let me know if you find this checklist useful in your reviews.

Automate tasks in software testing

When the term automation is mentioned, it is common for people to think of automated tests. But, time and effort can also be saved by automating other tasks such as generating test data and reporting test results automatically. A few factors should be considered for effectively deciding whether to automate a task or not. Last year, we had fruitful discussions on this topic within the STS group.

Let us see some candidate tasks for automation. Testers spend a lot of time on these tasks. If such tasks are fully or partially automated, the saved time can be spent on testing the system.

1. Test identification based on risk or other factors

List all the available test cases along with related features, components and priorities. The automation can take the inputs e.g. impacted features and generate the list of test cases that should be executed on the new application build.

2. Test estimation

Automation can be used to help estimate the test effort and duration. This can be done based on the preferred estimation approach. Whether by querying the historical data for actual efforts and durations, or by applying a custom formula to estimate test effort and test duration.

3. Generation of test data

Build a library of business rules for your test data. Build the initial seed data manually. The automation can use the seed data and generate test data based on the chosen business rules. Since test data generation is usually consumes much time, the automated generation can be performed ahead of time. Then, the pre-generated test data can be used directly during test execution.

4. Generation of bug reports

This automation can be built into the test automation framework. Whenever an automated test script confirms an error, it logs into the bug tracking system and reports a bug with required information. Such as bug title, steps to reproduce, test data used, environment used and so on.

5. Generation of test reports

This automation can execute queries against the test management system (and bug tracking system, if different) and generate test reports. Even distribute them by email or publishing to a website.

6. Release notes preparation

Release notes contain both static and dynamic data. This automation can execute queries on the test management system to retrieve the dynamic data such as features passed, bug fixes passed and known bugs.

Before prioritizing the automation development, always analyze the following factors.

a. Degree of automation achievable

b. Skills required to automate

c. Effort required to automate

d. Number of proposed users

e. Effort required to train users

f. Effort required to execute automation

g. Effort required to maintain automation

h. [Important] Manual effort saved

In the future, I see a number of such tasks automated with the help of vendor tools or bespoke in-house automation.

Let me know if you liked this post. I would love to know your thoughts on this topic.

Image Courtesy: Ambro / FreeDigitalPhotos.net