Tips to build your test data

If you want to know what is test data, view my video on Test Data. Now, let us see the different types of test data:

Test data types

a. Application configuration data
Your application likely needs test data to function (or even launch). Examples of application configuration data include the information to connect to database(s), admin user name and password and server information (e.g. email server) in order to send email notifications.

b. Application data
Other than the above, an application may require application specific data. Examples of application specific data are the menus/ links (and their hierarchies) and detail item information (e.g. item name, item details and item price).

c. Customer or user data
The users creating your application may create their own data within the application. Examples of customer or user data are user details (name, address and other personal details), user searches and user transactions (items browsed and purchased).

Tips to build test data

The questions that you should consider when creating or sourcing the test data for your tests are:

1. Does your application handle blank test data?
In case, your application does not find a particular test data, it should not crash. It should either display an informative (just the required information and no more; otherwise it may be security risk) message or it should display no data but working functionality. For example, if the information to connect to a database is missing, the application should throw an appropriate error message. If the user has not entered any required details, it should prompt the user to enter those details first.

2. Does your application handle invalid test data?
Data input to an application may be invalid in terms of data type (e.g. character data provided instead of the required numeric data), data size (insufficient or excessive size), outside the valid data range or just incorrectly formatted (dots instead of spaces). There are several approaches to deal with invalid test data. An application may reject the invalid test data, provide valid options for the user to select or attempt a conversion to valid data. In any case, invalid data should not be accepted by your application.
Even if the application handles new invalid data correctly, you should test the existing data for invalids. See the articles on database testing for details.

3. Does your application handle valid test data across the entire range of such data?
Of course, your application should accept valid data. However, using a single test data value for a range is not sufficient. Your test data should include the boundary values and several values (you should ask yourself how many values are reasonable) in between.

4. Does your application store confidential user data?
If you use a copy of the current production data, you could be exposing confidential user information to unauthorized persons. You could consider using only the application configuration data and application data from production. Another way to prevent exposure of confidential user information is to obfuscate the customer data from production.

5. Does the application handle the data volumes present or likely in production?
If your test data includes only a handful of data values and there are thousands of data values in production, your testing will not be realistic. You should match your test data quantity with that in production. Further, for realistic tests, your test data values should not be simple duplicates but vary as in production.

6. Is new data released into production from time to time?
You should test the data being released into production independently as well as with your application. After testing, you should update your test database with the new test data released to production.

What types of testing should you perform?

There are many kinds of testing that one hears about. You would likely know about many types of testing e.g. white box testing and black box testing, unit testing, integration testing, system testing, system integration testing and so on. The types of testing that would be performed on the application are usually identified in the test planning phase. If you wish to know which kinds of testing are suitable in your project, consider the following questions:

1. Do you have intimate knowledge of the design, logic and code of your application?

If you do, you may want to opt for white box testing. Otherwise (or additionally), you can perform black box testing. You can base your black box testing on the requirements and your knowledge of how your application should function.

2. What types of testing are common in projects?

Most applications are written not as single lengthy source code but rather as a collection of units (e.g. routines, functions or modules). In unit testing, these units are tested in isolation. In integration testing, the interactions between the units are tested.

Further, every application has some functionality. You test the functional requirements in functional testing.

Usability testing is used to determine if the application is easy to learn and use. However, usability testing is more commonly performed on applications with a large user base.

3. Does one test run/ test cycle on your application take substantial time?

If so, you may want to perform sanity testing/ smoke testing to determine if the application is stable and functioning well enough to deserve lengthy or detailed tests.

4. Will your application be used by more than one user? Will it hold or display confidential data?

If so, you should perform security testing not only on your application but also on the application infrastructure (e.g. application servers, database and network).

5. Does your application need to satisfy specific non-functional requirements?

You can group the non-functional requirements. Each group may require a special kind of testing. For example, if yours is a web application that is supposed to run on multiple browser versions, you need to do browser compatibility testing. If your application needs to respond within specific times, you need to do performance testing.

6. Will the user need to install the application before using it?

If so, you should perform installation testing (install and uninstall of the application on each supported platform).

7. Is it a new application or an upgrade?

If it is an upgrade, testing the new features of the application is not enough. You need to ensure that the previously working features are still working correctly. You need to do regression testing.

8. Does your application communicate with other applications/ systems?

System testing (testing the application as a whole) is applicable to every application. In case your application interacts with other applications or systems, you should also perform system integration testing.

9. Do you have an independent team (excluding developers and testers) who can test the application?

If so, they can perform alpha testing on (major) releases before completion.

Sample database test plan

In my video, Database Testing Explained, I referred to the sample database test plan below. First, view the video on Database Testing and then read on. First are some tips to create a good database test plan:

1. Database testing can get complex. It may be worth your while if you create a separate test plan specifically for database testing.

2. Look for database related requirements in your requirements documentation. You should specifically look for requirements related to data migration or database performance. A good source for eliciting database requirements is the database design documents.

3. You should plan for testing both the schema and the data.

4. Limit the scope of your database test. Your obvious focus should be on the important test items from a business point of view. For example, if your application is of a financial nature, data accuracy may be critical. If you application is a heavily used web application, the speed and concurrency of database transactions may be very important.

5. Your test environment should include a copy of the database. You may want to design your tests with a test database of small size. However, you should execute your tests on a test database of realistic size and complexity. Further, changes to the test database should be controlled.

6. The team members designing the database tests should be familiar with SQL and database tools specific to your database technology.

7. I find it productive to jot down the main points to cover in the test plan first. Then, I write the test plan. While writing it, if I remember any point that I would like to cover in the test plan, I just add it to my list. Once I cover all the points in the list, I review the test plan section by section. Then, I review the test plan as a whole and submit it for review to others. Others may come back with comments that I then address in the test plan.

8. It is useful to begin with the common sections of the test plan. However, the test plan should be totally customized for its readers and users. Include and exclude information as appropriate. For example, if your defect management process never changes from project to project, you may want to leave it out of the test plan. If you think that query coding standards are applicable to your project, you may want to include it in the test plan (either in the main plan or as an annexure).

Now, let us create a sample database test plan. Realize that it is only a sample. Do not use it as it is. Add or remove sections as appropriate to your project, company or client. Enter as much detail as you think valuable but no more.

For the purpose of our sample, we will choose a database supporting a POS (point of sale) application. We will call our database MyItemsPriceDatabase.

Introduction

This is the test plan for testing the MyItemsPriceDatabase. MyItemsPriceDatabase is used in our POS application to provide the current prices of the items. There are other databases used by our application e.g. inventory database but these other databases are out of scope of this test.

The purpose of this test plan is to:

1. Outline the overall test approach

2. Identify the activities required in our database test

3. Define deliverables

Scope

We have identified that the following items are critical to the success of the MyItemsPriceDatabase:

1. The accuracy of uploaded price information (for accuracy of financial calculations)

2. Its speed (in order to provide quick checkouts)

3. Small size (given the restricted local hard disk space on the POS workstation)

Due to limitation of time, we will not test the pricing reports run on the database. Further, since it is a single-user database, we will not test database security.

Test Approach

1. Price upload test

Price upload tests will focus on the accuracy with which the new prices are updated in the database. Tests will be designed to compare all prices in the incoming XML with the final prices stored in the database. Only the new prices should change in the database after the upload process. The tests will also measure the time per single price update and compare it with the last benchmark.

2. Speed test

After analyzing the data provided to us from the field, we have identified the following n queries that are used most of the time. We will run the queries individually (10 times each) and compare their mean execution times with the last benchmark. Further, we will also run all the queries concurrently (in sets of 2 and 3 (based on the maximum number of concurrent checkouts)) to find out any locking issues.

3. Size test

Using SQL queries, we will review the application queries and find out the following:

a. Items which are never used (e.g. tables, views, queries (stored procedures, in-line queries and dynamic queries))

b. Duplicate data in any table

c. Excessive field width in any table

Test Environment

The xyz tool will be used to design and execute all database tests. The tests will be executed on the local tester workstations (p no.s in all).

Test Activities and Schedule

1. Review requirements xx/xx/xxxx (start) and xx/xx/xxxx (end)

2. Develop test queries

3. Review test queries

4. Execute size test

5. Execute price upload test

6. Execute speed test

7. Report test results (daily)

8. Submit bug reports and re-test (as required)

9. Submit final test report

Responsibilities

1. Test lead: Responsible for creating this test plan, work assignment and review, review of test queries, review and compile test results and review bug reports

2. Tester: Responsible for reviewing requirements, developing and testing test queries, execute tests, prepare individual test results, submit bug reports and re-test

Deliverables

The testers will produce the following deliverables:

1. Test queries

2. Test results (describing the tests run, run time and pass/ fail for each test)

3. Bug reports

Risks

The risks to the successful implementation to this test plan and their mitigation is as under:

1.

2.

3.

 

Copyright © Software Testing Space

 

Approval

       Name        Role        Signature        Date

1. ____________________________________________________________

2. ____________________________________________________________

3. ____________________________________________________________