Software test effort estimation is a key skill required of a Test Manager (for multiple projects), a Test Lead (for the activities of the testing team) or a Tester (for own activities). In this post, I will not explain what software test effort estimation is or how to do it. If you need to know these things, view my video on Test Estimation techniques, formula example and Questions and Answers.
In this post, I will pose relevant questions regarding effort estimation in software testing. These questions are useful to guide yourself when you do estimation. They help clarify thoughts during discussions among project stakeholders. Also, these questions are useful in interviews for various testing positions. I have also discussed these questions and their answers in the above video on Test Effort Estimation.
Now, you may know that once the effort estimates are agreed, there is a (sometimes unsaid) expectation of commitment to these estimates. Therefore, generating a fair effort estimate in software testing is a serious task. Get the effort estimate loose and the resources and time would be under-utilized. Get the effort estimate too tight and there is a risk of stressing the resources and, in extreme cases, even failure to meet project deadlines. With this in mind, let us see the questions.
Image courtesy of imagerymajestic / FreeDigitalPhotos.net
In this post, I will pose relevant questions regarding effort estimation in software testing. These questions are useful to guide yourself when you do estimation. They help clarify thoughts during discussions among project stakeholders. Also, these questions are useful in interviews for various testing positions. I have also discussed these questions and their answers in the above video on Test Effort Estimation.
Now, you may know that once the effort estimates are agreed, there is a (sometimes unsaid) expectation of commitment to these estimates. Therefore, generating a fair effort estimate in software testing is a serious task. Get the effort estimate loose and the resources and time would be under-utilized. Get the effort estimate too tight and there is a risk of stressing the resources and, in extreme cases, even failure to meet project deadlines. With this in mind, let us see the questions.
1. What data would you gather and analyze before starting test effort estimation?
I would need the scope of work (i.e. the size of the system, types of testing required, test strategy, planned test cycles, "non-test" activities e.g. test data generation, defect logging and tracking, triage meetings etc.). The scope of work is better defined in a Work Breakdown Structure (WBS) for the project. Also I would need historical data of similar projects (as similar as possible in size, development life cycle, technology and team composition). This history data may be available in the company records or can be gathered from the team who worked in the similar project.
It is quite possible, in fact common, for some of the above data to be not available. But in any case, the scope of the work should be defined and agreed by the stakeholders (i.e. the client/ business, management and the project team) before test effort estimation is done.
I would need the scope of work (i.e. the size of the system, types of testing required, test strategy, planned test cycles, "non-test" activities e.g. test data generation, defect logging and tracking, triage meetings etc.). The scope of work is better defined in a Work Breakdown Structure (WBS) for the project. Also I would need historical data of similar projects (as similar as possible in size, development life cycle, technology and team composition). This history data may be available in the company records or can be gathered from the team who worked in the similar project.
It is quite possible, in fact common, for some of the above data to be not available. But in any case, the scope of the work should be defined and agreed by the stakeholders (i.e. the client/ business, management and the project team) before test effort estimation is done.
2. What approach would you use to create a test effort estimate?
There are several approaches that can I can use for test effort estimation. It depends on the available data and the time in hand to generate the estimates (which, in turn, may or may not affect the accuracy of the estimates). Some of the common effort estimation approaches include the using the software size, using the WBS (include "non-test" activities in the WBS, if missing), use project history data or if available, use the company's estimation software and estimation models. If (some of) the above data is not available, I could use my expert judgment (or judgment of more than one person). It is better to use at least two approaches. This way, I can see and address any obvious problems in the effort estimate.
There are several approaches that can I can use for test effort estimation. It depends on the available data and the time in hand to generate the estimates (which, in turn, may or may not affect the accuracy of the estimates). Some of the common effort estimation approaches include the using the software size, using the WBS (include "non-test" activities in the WBS, if missing), use project history data or if available, use the company's estimation software and estimation models. If (some of) the above data is not available, I could use my expert judgment (or judgment of more than one person). It is better to use at least two approaches. This way, I can see and address any obvious problems in the effort estimate.
3. Which approach would you prefer if the test effort estimates are needed quickly?
There may be business situations requiring estimates to be generated quickly. In such a case, I would prefer estimating activities at a summary/ group level. Then use another estimation approach to know if the estimates seem okay. Further, I would add some buffer, of say 5%, of overall effort, to use just in case there are unforeseen tasks to perform in the test.
There may be business situations requiring estimates to be generated quickly. In such a case, I would prefer estimating activities at a summary/ group level. Then use another estimation approach to know if the estimates seem okay. Further, I would add some buffer, of say 5%, of overall effort, to use just in case there are unforeseen tasks to perform in the test.
4. Which approach would you use to get accurate test effort estimates (for example, as required by a services company to quote in a fixed bid project)?
If possible, I would like to use the bottom-up approach for creating effort estimates for all tasks in the WBS. The individual effort estimates roll up to give the effort estimates for each group task. In order to refine the effort estimates further, some work items may be sampled and then executed. The actual effort spent on performing these work items can be used to improve the effort estimates.
If possible, I would like to use the bottom-up approach for creating effort estimates for all tasks in the WBS. The individual effort estimates roll up to give the effort estimates for each group task. In order to refine the effort estimates further, some work items may be sampled and then executed. The actual effort spent on performing these work items can be used to improve the effort estimates.
5. What is the two-estimate approach?
It is possible that the client has a fixed budget for which they can be billed. Internally, the contracting person or company may have reasons to work more hours than for which they bill the client. In such a case, the effort estimates with the client and the contracting person/ company would be different.
It is possible that the client has a fixed budget for which they can be billed. Internally, the contracting person or company may have reasons to work more hours than for which they bill the client. In such a case, the effort estimates with the client and the contracting person/ company would be different.
6. What data would you analyze to confirm or revise your test effort estimate?
I would gather and analyze the actual time data logged by the team members against project tasks. It's quite possible that the team spends more or less time than estimated on individual project tasks. The reasons for doing so should be understood so that effort estimates of similar tasks yet to be performed can be refined, if required.
I would gather and analyze the actual time data logged by the team members against project tasks. It's quite possible that the team spends more or less time than estimated on individual project tasks. The reasons for doing so should be understood so that effort estimates of similar tasks yet to be performed can be refined, if required.
7. Is any supporting data required with your test effort estimate?
It is not possible to get buy-in for an effort estimate without giving the estimation approach used. Therefore, I would outline the effort estimation approaches used to generate and support the estimates. Further, I would provide the assumptions used while creating the estimates.
It is not possible to get buy-in for an effort estimate without giving the estimation approach used. Therefore, I would outline the effort estimation approaches used to generate and support the estimates. Further, I would provide the assumptions used while creating the estimates.
8. What risks do you see to render a test effort estimate inaccurate?
There are an infinite number of risks that could throw an effort estimate off. But the important risks include scope risks (e.g. important activities missed in estimation, feature creep, last minute system changes), resource risks (e.g. resource not committed to the project or not productive), quality risks (e.g. build rejects) and technical risks (e.g. test environment not available, test deliverables not accepted).
Despite risk mitigation measures in the testing risk management plan, if a particular risk turns true it should be communicated to the project stakeholders. It is possible that someone within the team or outside it has a solution to this issue. If there is no solution, the team has to absorb the risk impact.
There are an infinite number of risks that could throw an effort estimate off. But the important risks include scope risks (e.g. important activities missed in estimation, feature creep, last minute system changes), resource risks (e.g. resource not committed to the project or not productive), quality risks (e.g. build rejects) and technical risks (e.g. test environment not available, test deliverables not accepted).
Despite risk mitigation measures in the testing risk management plan, if a particular risk turns true it should be communicated to the project stakeholders. It is possible that someone within the team or outside it has a solution to this issue. If there is no solution, the team has to absorb the risk impact.
9. When would you revise the test effort estimates?
I would revise the estimates Whenever there are material changes in the project. Examples of such changes are scope changes (e.g. new features added to the system under test, new types of testing like load testing needed) and resource changes (e.g. team member replaced in the team).
I would revise the estimates Whenever there are material changes in the project. Examples of such changes are scope changes (e.g. new features added to the system under test, new types of testing like load testing needed) and resource changes (e.g. team member replaced in the team).
10. How do you improve your estimation skills?
Like other skills, practice makes perfect. Comparing effort estimate data and actual effort data leads to continuous improvement. A big opportunity for improvement is project reviews (gate/ phase-end reviews and project completion reviews). Lastly, my estimation skills improve with doing estimation on more and more projects.
Like other skills, practice makes perfect. Comparing effort estimate data and actual effort data leads to continuous improvement. A big opportunity for improvement is project reviews (gate/ phase-end reviews and project completion reviews). Lastly, my estimation skills improve with doing estimation on more and more projects.
Image courtesy of imagerymajestic / FreeDigitalPhotos.net
good questions and examples.
ReplyDeleteThanks for this posting!
ReplyDeleteNice & very informative blog.
Software Testing Training Noida
Expected to form you a next to no word to thank you once more with respect to the decent recommendations you've contributed here.
ReplyDeleteEstimates
Your blog is excellent, and you have provided very important information through this blog. I was looking for a similar blog for a long time that can clear my doubts. That's why I want to thank you, and I wish you to keep writing similar blogs in the future. Apart from this, if you ever need a Manual Software Testing Company then you can get all the information on this website.
ReplyDelete