How to encourage testing within your organization

On interacting with testers in various forums, one thing amazes me. Even today, there are organizations developing software but with little or no testing processes. It is not that they do not test at all. The developers test their code but there is no "tester" testing. Or testing is not institutionalized. If you work for such an organization and want to encourage testing, here are a few tips for you to consider.

When Manual Testing is better than Automated Testing

Manual testing is testing done by the tester using his intellect but without the use of automated testing tools. Manual testing has possible disadvantages. It can be expensive in effort, have human mistakes or just be boring. Even so, manual testing continues to be very popular because of its powerful advantages. Since manual testing is done by a human being and not a program, the test design and test execution can be changed very quickly. Manual testing is done using the creative and analytical skills of the tester and it requires no investment in software testing tools. Now, let us see the situations where manual testing is the better option due to technology limitations of today.

How video game testers work?

Video games or computer games are available on a number of devices such as consoles, computers and mobiles. They are enjoyed by countless number of people. Video games have existed for a long time now. However, with artistic and technological advances, the video games have become very complex. This complexity needs serious game testing. Game testing is intense work but fulfilling because the game industry is growing rapidly. Game testing involves a number of tasks. It also needs a passion for playing games. The role of a game tester is to provide objective information about the game. What are the good things about the game? What things about the game should change? Are there any bugs or glitches in the game? Does the game work with good performance?

Learn from my video, How to Test Games Software or read on.

Game Testing

The main objective of a game is to have fun. The definition of fun is different for different gamers. The game tester has to know the target users and what they find enjoyable. Other than the fun element, a game may have objectives like developing cognitive/ motor skills or gamer education. Moreover, a game may have a background, a timeline, a plot, characters, theme, graphics, events, scores, levels, paths and other elements. The game tester needs to test each of the above elements with respect to the game objectives and contemporary games in the market. From the first testable code through to release, the game developers provide game builds to the game tester. The game tester should run the following tests.

Installation testing
Unless it is a (totally) web-based game or an embedded game, there is some software that the player needs to install on their device first. The installation process has a series of steps that need to be tested on each supported device configuration used by gamers. The game tester should run the steps on each supported device configuration. He should also check the completeness and accuracy of the content in the installation guide.

Feature and Functional testing
This is the most important game test. Typically, the game would have a number of features. The game tester should test the game features fully. The items that need to be tested include basic game play like start/ suspend/ resume/ stop/ start a saved game, game options, progression from one level to the next, correct scores and correct working of the controls. Then, the game tester should test if the game is true to its background, timeline, theme, plot, characters and other elements. The game tester should also test game events, if any and concurrency of multiple gamers in multi-player games.
In order to find the bugs, the game tester should be creative by taking different paths and performing unexpected actions in the game. In addition to finding the bugs, the game tester should estimate how well the game meets its objectives. Also, identify improvement opportunities in the game.

User Interface (UI) testing
This is also an important game test. UI testing includes testing of the graphic elements (e.g. characters in the plot, backgrounds, floors, objects and people) and the content (both viewable and audible). If the game is targeted for multiple languages and cultures, the game tester should test it in each supported language.

Performance testing
A novel story line, incredible graphics, sounds, music and numerous well-integrated features of a game may still fail to satisfy the player if the game is too slow or freezes. The game tester should test the speeds of operation of the game on supported device configurations. The game tester should also play the game at extremes, for example, run a complete game without any pauses at all or run a game continuously for 24 hours or keep on increasing the number of players if the game is multi-player. These tests show the performance bugs.

Alpha test and Beta test support
Once the game development is complete, it is released to few reviewers (alpha test). The game tester should explain the game details to the alpha testers and get their feedback. Then gather alpha test bugs or improvement opportunities, confirm these and pass the valid bugs and change requests to the developers. The game tester should test the game changes and bug fixes before the new build is given to the alpha group.
Once the game is released to a limited number of gamers for their review (beta test), the game tester should confirm the bugs reported by beta testers. Then provide these to the developers. The game tester should test the bug fixes before the final release of the game to market.

Software Testing Estimation

Software testing effort estimation is a skill required of a Test Lead or a Test Manager. However, test effort estimation is not a skill that one can learn quickly. It requires understanding of several key concepts and practice.  In the previous post on estimation techniques, we have seen six approaches to get software test effort estimates. In this post, we will look at one possible software testing estimation formula. Before using it, you need to customize it according to your test process. This formula is explained with an example in my video on Test Estimation with formula example.

Software Testing Estimation Formula

Task	Usage notes	Cumulative Effort Estimate(hours)
QA/test environment preparation	Estimate this based on hardware installation, software installation and deployment of builds as T1	T1
Testing of n requirements	Estimate testing time per requirement, T2, based on the detail of requirements e.g. T2 can be 0.5 if requirements are detailed or up to 4 if the requirements are high level.	T1 + T2 * n
Negative and exploratory testing	Use, say, 40% extra testing time per requirement, to cover the paths other than the “happy path”, and exceptional and invalid test data inputs.	T1 + T2 * n * 1.4
Re-testing due to requirement correction or changes	Estimate factor T3, based on requirement correctness and stability e.g. T3 can be 0.1 if requirements are well put and stable or 0.5 if requirements have multiple issues or are very dynamic. Let us use T3 as 0.2.	T1 + T2 * n * 1.6
Logging defects and re-testing fixes	Estimate the number of defects as T4. Estimate time to log defect, say 0.25 and time to re-test a fix, say 0.25	T1 + T2 * n * 1.6 + T4 * 0.25 + T4 * 0.25
Regression testing	Estimate this effort based on the impact of requirements implemented, defects and depth needed as T5.	T1 + T2 * n * 1.6 + T4 * 0.5 + T5
Reserve	Having this reserve helps to provide time to complete unforeseen tasks. Estimate the reserve as T6 between 5 and 10% of overall effort	(T1 + T2 * n * 1.6 + T4 * 0.5 + T5) * (1 + T6)
Communication	Estimate the daily effort in hours on meetings and reporting as T7. Multiply it by team members T8 and number of testing days T9.	(T1 + T2 * n * 1.6 + T4* 0.5 + T5) * (1 + T6) + T7 * T8 *T9

Now let us see an example. Assuming the following:
Test environment preparation time, T1 = 2 hours
Number of requirements, n = 100
Time to test each requirement, T2 = 0.5 hours
Number of defects, T4 = 10
Regression testing time, T5 = 8 hours
Reserve, T6 = 5% i.e. 0.05
Daily time spent on meetings and reporting, T7 = 0.5 hours
Number of testers, T8 = 2
Number of testing days, T9 = 5
(T1 + T2 * n * 1.6 + T4* 0.5 + T5) * (1 + T6) + T7 * T8 *T9 becomes
(2 + 0.5 * 100 * 1.6 + 10* 0.5 + 8) * (1 + 0.05) + 0.5 * 2 *5 = 104.75 hours
This time includes the time to set up the test environment, positive/ negative/ exploratory testing of requirements, re-testing requirement clarifications/ changes, logging defects and re-testing fixes, regression testing, meetings and reporting.

There are several ways to confirm your software testing effort estimate.  Use your judgment to find if it looks too low, right or too high. Compare it with the actual testing efforts data for previous projects. Find out if it is 25% - 30% of development effort (do not know why but I have noticed that this range seems acceptable to project managers). You can also ask other managers to review your estimate. Once you have arrived at the test effort estimate for your project and have convinced the stakeholders that it is a reasonable estimate, track the actual progress in your project constantly to see if it is in line with your estimate.

Significant deviation from software testing effort estimate may have financial implications. If you end up charging for more testing effort, the client would not be happy. Alternatively, if it a fixed-price project, you company would not be happy with reduced profitability that comes with higher actual effort. Realistic estimation is important to pay attention to coming up with the suitable formula, estimate the factors realistically and confirm your estimate by using alternative approaches.

Tester role in agile software testing

Agile software development is very popular in the current software industry. Agile places emphasis on constant communication and collaboration within the team and with the customer. Its objective is to develop increasingly functional software in a dynamic requirements landscape. Agile projects run differently from projects using Waterfall model or Iterative model. Testers working in Agile projects should be familiar with their role therein.