How to do end to end exhaustive testing?

Testing a software application (except maybe a very simple program a few lines long) may well be an impossible task due to large number of:

1. All possible inputs

2. All possible input validations

3. All possible logic paths within the application

4. All possible outputs

5. All possible sequences of operations

6. All possible sequences of workflows

7. All possible speeds of execution

And the above for just with a single user

8. All combinations of types of users

9. All possible number of users

10. All possible lengths of time each user may operate the application

And so on (we have not even touched the types of test environments on which the tests could be run).

However, it is possible to exhaustively execute your test suite using the following tips:

1. Your test suite should have test cases covering each documented requirement. Here my assumption is that each requirement is documented clearly.

2. The test cases should be specific, concise and efficient. Each test case should have clear and unambiguous steps and expected results.

3. The configuration data, input test data and output test data should be clearly specified.

4. You should have a clean and stable test environment in which to execute your test suite.

5. In a perfectly working application, it should be possible to execute each test case in the suite.

6.. Each confirmed bug (found during testing or found by the client) should result in another test case written or an existing test case updated.

7. Important: You should not assume the correctness and completeness of your test suite by yourself. Review of such test suite by peers, business people, managers, clients and users may provide you valuable inputs to correct your test suite.

8. Discipline in maintaining your test suite and executing it would go a long way in preventing bugs leaked to the clients/ users of your application.

Conventional wisdom applied to software testing

Here are the conventional English proverbs as I see applied to the field of software testing:

1. A journey of a 1000 miles begins with a single step.

A test run of a 1000 test cases begins with a sanity test.

2. Genius is 10% inspiration and 90% perspiration.

Software testing is 10% test preparation and 90% test execution.

3. As you make your bed, so you must lie in it.

As you write your test cases (good or poor; complete or incomplete), so you must execute them.

4. A bird in hand is worth two in the bush.

A confirmed bug is worth two suspects.

5. A chain is no stronger than its weakest link.

The quality of an application is no better than that of its most buggy component.

6. Discretion is the better part of valor.

Review is the better part of executing a task.

7. Empty vessels make the loudest noise.

People who least understand software testing speak of it in the most black-and-white terms.

8. An ounce of prevention is worth a pound of cure.

One bug fixed in the beginning is worth 16 updates.

9. A penny saved is a penny gained.

One test case re-used is a test case written.

10. Where there's a will, there's a way.

Where there's an application, there are bugs yet to be found.

Hope you enjoyed these :). If you like any other English saying, let me know. I will translate it to software testing.

Checklist for application release notes

It is a norm to distribute release notes along with a software release. Software release notes are similar to the product literature that you get when you buy a physical product. You may vary the contents depending on whether the software release is a major release or a minor release, an upgrade to the application or just an update, customized for a specific customer or available generally. Find answers to the following questions to test the release notes that you receive before software testing.

Main information

1. Does it mention the application name and the correct version number?
2. Does it correctly list the features released and/ or bugs fixed in the software?
3. Does it mention the correct software vendor name and date of the release?
4. Does it mention the release numbers it requires as prerequisite or the release numbers it supersedes?
5. Does it mention each of the available distribution modes (e.g. website download, email and disc)?
6. Does it mention the correct distributed media (e.g. install package, executable programs or scripts) in which the software release exists?

Release feature information

1. Does it explain each new feature (summary, benefits and details of how it works)?
2. Does it explain bug fixed in this release?

Supporting information

1. Does it list the minimum and recommended hardware, software and network requirements to deploy and use the release?
2. Does it provide steps to backup the existing configuration and user data in the system?
3. Does it provide detailed steps to install the release?
4. Does it provide steps to restore the configuration and user data of the previous release?
5. Does it provide the contact channels in case the software does not install or installs with problems?

Others

1. Is the release notes available in a file format that can be viewed by the customer?
2. Is the file format consistent with prior releases?