Wednesday, March 31, 2010

Software Test Automation Estimation (10+ factors which you should consider)

You may have seen my video, Test Estimation with formula example and questions and answers. In this post, we will discuss the factors that you should analyze in order to arrive at a realistic effort estimate for creating the test automation.

1. Knowledge transfer

Is your team new to the Application under test or would they need training or time with the application to be comfortable with it?
Is your team new to the chosen test automation tool(s)? Would they need training on the tool before they are productive with it?

2. Test automation environment

How long would it take to set up the test environment with the application and the tool for each team member in your test automation team?

3. The chosen tool's compatibility with your application

Do you need to perform a test tool evaluation before you begin automation?
How compatible is the chosen test automation tool with your application's technologies (e.g. does the tool recognize each type of object in your application, does the tool work fast with your application and so on)?

4. Test automation framework

Would you need to create a test automation framework from scratch?
Would you need to use an existing test automation framework? How simple or complex is it to learn to use this existing test automation framework?

5. Test cases to be automated

Would you have the test cases available for automation? Are these test cases automatable?

6. Size of the automation

Considering the size of the test case, the speed of your application and the speed of the chosen test automation tool, how long would it take to automate each test case?
By how much would the usage of your test automation framework affect the effort of automating each test case?

7. Test data requirements

Would test data be available or would your team need to generate own test data?
Where would the test data be located?

8. Types of testing required

What kind of unit tests would be performed on your test automation script?
What kind of integration tests would be performed on the integrated automation scripts?
What kind of tests would be performed to check the validity of test data?
Do you need to create automation only for functional tests? Or for other tests as well e.g. performance tests?

9. Supporting automation

Would your team need to create automation for non-testing activities (e.g. creating a lot of dummy data within the application or emailing test logs to users)?

10. Version control and test automation builds

How slow or fast is the version control?
How long would it take to integrate the test automation created by each team member? How frequently would the test automation build be created?

11. Reviews

What test automation items would be reviewed e.g. test scripts, test data?
What will be the frequency of these reviews?
How long would it take to complete the reviews and re-work?

12. Maintenance

How frequently would the base lined test cases be updated? How long would it take to update your test automation suite in line with these changes?
How frequently would your team receive an updated application build? (Keeping your framework and your chosen tool's capabilities in mind), how long would it take to update your existing test automation?