In software testing, test automation involves automatic execution by your computing device of test cases which would otherwise be executed manually by a tester. Obviously, test automation has to be created for such test cases before automated execution can be done. If you have limited time (as always) to create test automation, you may be wondering which of your test cases you could automate with the least effort. The purpose of this post is to make you aware of several considerations before you select the test cases for test automation. But first, let us see how test automation works.
The test automation code for a test case is a program. Now this program for the test case may be generated automatically by a test automation tool (the test automation engineer has to show the tool the operations and validations and likely edit the generated code). Or the program may be written by a test automation engineer by hand (in a development environment). This program contains instructions or statements for the following:
a. Achieve the pre-requisite conditions (e.g. open the application under test, log in as a valid user etc.).
b. Navigate to and identify the application object (e.g. a specific window, a specific button) on which the test steps are to be performed.
c. On the application object, perform the operations (e.g. click, enter data, select a file) that are listed in the test steps of the test case.
d. Identify the application object (e.g. search grid, open window, message popup box) that contains the actual results.
e. On this application object, read the properties that contain the actual results.
f. Compare the actual results with the expected results listed in the test case.
g. Store these comparison (test) results somewhere (e.g. a log, a file or a database).
h. Achieve the post-test conditions (e.g. log out, close the application or service).
i. If any statement fails in the steps above, store the error(s) somewhere (a file or a database).
a. Achieve the pre-requisite conditions (e.g. open the application under test, log in as a valid user etc.).
b. Navigate to and identify the application object (e.g. a specific window, a specific button) on which the test steps are to be performed.
c. On the application object, perform the operations (e.g. click, enter data, select a file) that are listed in the test steps of the test case.
d. Identify the application object (e.g. search grid, open window, message popup box) that contains the actual results.
e. On this application object, read the properties that contain the actual results.
f. Compare the actual results with the expected results listed in the test case.
g. Store these comparison (test) results somewhere (e.g. a log, a file or a database).
h. Achieve the post-test conditions (e.g. log out, close the application or service).
i. If any statement fails in the steps above, store the error(s) somewhere (a file or a database).
So, how do you select test cases that would take less effort to automate than other similar sized test cases? Consider the points below:
1. Technologies building your application
Creating test automation depends on identifying application objects, performing operations on them and reading their properties. A web application created using HTML, javascript and MySQL uses popular technology that is well understood. Perhaps it would take less effort to automate test cases of your web application with a known object model than to automate test cases of your old desktop application using old forms, a closed object model and an encrypted file database that few people understand.
2. User operations
Selecting a menu item, entering/ selecting text in an input field, clicking a button or scrolling the search results are operations on application objects. Panning a map, drawing freehand in a canvas or viewing an image painted on the screen are likely operations within an application object. It would probably take less time to automate a test case with object operations than to automate a test case (if at all) with image operations. This is because it may be simple enough for a human user to draw a signature or recognize an image but it is a complex task for test automation. Replicating such user operations may require graphics API calls or programming numerous low level mouse movements.
3. Details in the test cases
If a test case is missing details e.g. pre-conditions, certain test steps or test data, it would need to be completed before automating it. Click here for guidelines to create automation-friendly test cases. On the other hand, test automation effort may be started at once on a specific and detailed similar sized test case.
4. Existing test automation
If you have some test automation available, you may already have code (e.g. functions, routines or scripts) to execute some pre-conditions and perform some operations or validations in specific application modules. Therefore, it may be simpler to re-use some of the existing code to create test automation of test cases from the same application modules than to automate test cases from other application modules.
5. Test data format
Test data may be stored separately from the test case, in a CSV file, a Microsoft Excel file or a database. Or it may be listed along with the test case, in a .pdf file or Microsoft Word document. It may be easier to create automation to read a test data CSV file or a database table or view than to programmatically navigate to a custom location within a test case document and then read the test data in a custom format.
6. Test case revisions
If you are going to be spending effort other than testing manually on automating test cases, you would want to execute them more than once. Perhaps you would like to execute them in multiple test environment configurations or upon multiple application builds. In order to reduce the effort on revising the test automation, it may be better to create test automation for the (relatively more) stable test cases.
Hope that I have given you some food for thought on how to invest your limited time to get more return on your test automation creation effort.
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
1. Technologies building your application
Creating test automation depends on identifying application objects, performing operations on them and reading their properties. A web application created using HTML, javascript and MySQL uses popular technology that is well understood. Perhaps it would take less effort to automate test cases of your web application with a known object model than to automate test cases of your old desktop application using old forms, a closed object model and an encrypted file database that few people understand.
2. User operations
Selecting a menu item, entering/ selecting text in an input field, clicking a button or scrolling the search results are operations on application objects. Panning a map, drawing freehand in a canvas or viewing an image painted on the screen are likely operations within an application object. It would probably take less time to automate a test case with object operations than to automate a test case (if at all) with image operations. This is because it may be simple enough for a human user to draw a signature or recognize an image but it is a complex task for test automation. Replicating such user operations may require graphics API calls or programming numerous low level mouse movements.
3. Details in the test cases
If a test case is missing details e.g. pre-conditions, certain test steps or test data, it would need to be completed before automating it. Click here for guidelines to create automation-friendly test cases. On the other hand, test automation effort may be started at once on a specific and detailed similar sized test case.
4. Existing test automation
If you have some test automation available, you may already have code (e.g. functions, routines or scripts) to execute some pre-conditions and perform some operations or validations in specific application modules. Therefore, it may be simpler to re-use some of the existing code to create test automation of test cases from the same application modules than to automate test cases from other application modules.
5. Test data format
Test data may be stored separately from the test case, in a CSV file, a Microsoft Excel file or a database. Or it may be listed along with the test case, in a .pdf file or Microsoft Word document. It may be easier to create automation to read a test data CSV file or a database table or view than to programmatically navigate to a custom location within a test case document and then read the test data in a custom format.
6. Test case revisions
If you are going to be spending effort other than testing manually on automating test cases, you would want to execute them more than once. Perhaps you would like to execute them in multiple test environment configurations or upon multiple application builds. In order to reduce the effort on revising the test automation, it may be better to create test automation for the (relatively more) stable test cases.
Hope that I have given you some food for thought on how to invest your limited time to get more return on your test automation creation effort.
Image courtesy of Stuart Miles / FreeDigitalPhotos.net
Nice article
ReplyDeleteVery useful post and I think it is rather easy to see from the other comments as well that this post is well written and useful. I bookmarked this blog a while ago because of the useful content and I am never being disappointed. Keep up the good work..
ReplyDeletesoftware testing company
QA Outsourcing Sevices