Wednesday, March 17, 2010

What is the best place to store test data for your automated tests?

You need test data to execute your tests. The test data is used to provide inputs to the application under test and/ or verify the test results. Test data may be used to check if the application works normally with expected inputs, handles incorrect inputs gracefully and (optionally) if the application works with multiple test data values. You can source the test data from an existing data store, create the test data by hand or automate the creation of the test data. If you want to learn more, view my video tutorial on Test Data and its types.


Now, the question arises where should you store test data once you have generated it? There are numerous data stores possible. Go through the table below to see a comparison of the data stores.
Ease of setupMaintainabilityRe-useCost
Text filesGood
(but it is not simple to secure the text files and it is not possible to store images in them)
Poor
(it is easy to make mistakes while creating or editing the test data)
GoodExcellent
SpreadsheetsGood
(since you are likely quite comfortable with spreadsheets)
Average
(since you may end up with test data in multiple sheets of multiple spreadsheets)
GoodAverage
(requires at least the spreadsheet viewer to read the spreadsheet)
RDBMSPoor
(since you first need to design the table structure to store your test data)
Excellent
(due to permanent storage and the availability of tools to view and edit the test data)
Excellent
(if the test data design is generic enough)
Poor
(owing to the possible high cost of the RDBMS)
Test data management toolExcellent
(due to the features provided by the tool)
Average to Excellent
(depending on the features of the tool)
Poor
(possibly, if porting to another test data management tool)
Poor to Excellent
(depending on the cost of the tool license)
XMLAverage
(you need to know XML but it is good
for defining hierarchies)
Average
(debugging test data may be challenging)
GoodExcellent
Application configuration filesGood
(you can take the help of developers to set up your test data)
Poor
(due to the presence of other data that is related to application settings)
Poor
(if porting to other applications under test)
Poor to Excellent
(depending on the cost of the development license)
Now, you know the different types of data stores for your automated tests.