Game Testing: How to test game software?

Games are available on a number of devices such as computers, mobile handsets, television sets and so on. In this post, I am going to focus on testing the software of a game that runs on a computer. See the video, Game Testing for Game Tester, or read on...

When you plan to test a software application, you tend to think about the types of testing required. Similarly, when you plan to test gaming software, you should identify the types of testing required. The common types of testing for game software are listed below. Keep in mind that the objectives of a game are quite different from the objectives of a business application. For example, the objectives of a business application could be to provide the given user the features to easily, reliably and accurately perform certain business tasks. In the case of a game, the main objective is having fun. There may be other objectives such as appealing to different types of player demographics, educating the players or developing team skills. Therefore, a game may be required to provide the following:
1. An intriguing or complex plot
2. Realistic/ delightful graphics (including characters, backgrounds and equipment) and sounds
3. Random events (to keep the player interested)
4. Little known facts (to educate the player)
5. Facilitate players to work as teams (In case of multi-player games)
Therefore, the following types of testing should be seen in the light of the given game's objectives.
1. Installation testing
Unless it is a (totally) web-based game, there would be some software that the player needs to install on their computer first. The installation process has a series of steps that should be tested on the common computer configurations (each configuration specifies data like the Processor, RAM, Hard Disk, Display, Display RAM, operating system etc.) used by players.
You should design test cases for the installation steps and then execute these test cases on each of the common configurations.
You should also check the completeness and accuracy of the content in the installation guide. This may be done just once (on the most common computer configuration).

2. Feature testing
This is the most important type of game testing. Typically, the game would have a number of features. You should aim to cover the game's features as exhaustively as feasible. An efficient way to accomplish this is to design test cases in a number of ways, such as:
a. Detailed test cases - These test cases are suitable to cover obvious features e.g. game options, progression from one level to the next, correct working of the controls, start/ suspend/ resume/ stop game etc.
b. Task based test cases - These test cases are at a higher level of detail. They take into account the objectives a player could have e.g. go through each level and win the game, choose and old saved game and continue it etc.
c. Test matrices - These test cases are useful when there are different features for different player statuses e.g. a particular weapon is available to the player only after clearing a particular level, the player can go through an obstacle only when they have collected a particular item before. A test matrix is drawn in the form of a table. The rows could then represent the features and the columns the player statuses (e.g. current level).
Executing the different types of test cases may still not be sufficient to discover a majority of the defects. You should perform exploratory testing. In simple words, this would mean play the game and perform interesting tasks that you think would reveal defects.

3. UI testing
Since games are typically based on user interfaces, this is also an important type of game testing. UI testing includes testing of the graphic elements (e.g. characters in the plot, backgrounds, objects in the foreground) as well as the content (both viewable and audible). An efficient way to test the UI is to first list the checks desired for every graphical element and content type. Then check each item in your list in every logical division of the game (e.g. each screen and each level).

There is an important thing that you should keep in mind regarding UI testing. It is possible that your game is released in multiple languages and cultures. If so, you should create a matrix (a table) listing all the supported languages. Then you should take the help of a speaker of a given supported language to check the content. You should get the content checked in each supported language at least once.

4. Performance testing
A novel story line, incredible graphics and sounds and numerous well-integrated features of a game may still fail to satisfy the player if the game becomes too slow or freezes up. You can check the speeds of operation of the game on common computer configurations. A good way to do this is to identify the common tasks a player is likely to perform. Then you should determine the acceptable times for these tasks. These times are the goals. You should then perform each task in the game and note the time it actually takes to do so. Since there are a large number of tasks to be timed, you should automate the performance tests.
Another flavor of performance testing comprises testing your game at extremes. For example, run a complete game without any pauses at all or run a game continuously for 24 hours or keep on increasing the number of players if your game is multi-player. Running such tests would give you data on how the performance of your game degrades on high loads. If the game shows performance problems at a realistic load, it means that you have discovered a problem that should be fixed.

You now know about the main types of tests that you can run on a game. Remember that feature tests and UI tests are especially important. Wouldn't you like to use your knowledge to test some game software?

Software Testing Books

Occasionally, people ask me about some good books on Software Testing. I refer them to this list. They say that though these books build up a subject from the beginning, everyone could learn something from them.

Knowledge Transfer: What is the best way to do it?

We know that a lot of knowledge is required by the tester before starting any test related tasks (even test planning). The knowledge is gathered and assimilated by the tester before the project starts or just after the project starts. The process by which this knowledge is gained is called Knowledge Transfer. Knowledge transfer is quite visible when a new team member joins the team and the existing team members (including the business people or the client) spend time in communicating with the new member to bring him or her up to speed. In an industry with a fondness for acronyms, knowledge transfer is also called KT.

Knowledge transfer is critical for testing. If your knowledge transfer has gone haywire, you could end up with people questioning your abilities (or worse, you could be out of the project). Remember that your successful performance depends on a good knowledge transfer to you. View my Domain Knowledge for Testers videos for industry domain knowledge in several industries. For example, view the video on Banking domain knowledge.

Sometimes the knowledge transfer happens in a chaotic, incomplete or hurried way due to any of the following problems:
a. Incorrect identification of knowledge givers
b. Lack of motivation on part of the knowledge givers
c. Communication challenges between you and the knowledge giver
d. Problems due to remoteness
e. Lack of trust (especially damaging to knowledge transfer)

The approach for knowledge transfer should take care of the above problems. Further, the approach should attempt to transfer knowledge that is located obviously in the documentation and processes and not so obviously in the team's behaviors and minds.

The knowledge transfer should have the following phases. Phases 3, 4 and 5 may be done in parallel. Otherwise, it is best to execute the phases in the given order. Even though the knowledge transfer may be initiated and closed by someone else (e.g. the client or the project manager), you are directly impacted by the success of the knowledge transfer. Therefore, you should take an active part in the process.

1. Prepare
The key people possessing the (most) knowledge in the various areas e.g. application functionality, application infrastructure etc. should be identified. It is not sufficient to inform the knowledge givers about the task. The knowledge givers should be motivated to impart the knowledge to you. This is usually best accomplished if the performance of the knowledge giver is somehow linked to the knowledge gained by you.

The one or more mechanisms of knowledge transfer should also be decided:
a. Demonstrations
b. Discussions
c. Providing documents as required by you
d. Replying to questions asked by you
e. Mentoring
f. Instant messaging
g. Wikis

2. Understand the client
It is important for you to understand the client organization (their business objectives, the purposes for which they use the application and so on). If the client cannot impart the knowledge to you, a business person (e.g. the business analyst) may impart the knowledge about the client.

3. Get familiar with the application
This is an important phase of the knowledge transfer since you would be spending a major part of her time with the application. A person very familiar with the application may demonstrate the functions and features of the application to you. Further, you should get a hold of a demo installation of the application in which you may explore the functions and features.
You should also get the (correct versions of) application documentation. Examples of application documentation include the system requirements, functional requirements, installation/ operating manuals, help files/ user guides, test cases, test plans and so on. This is the time to observe the application, read the documentation and ask as many questions are required to learn the nuances of the application.

4. Get familiar with the development, project management and test processes and systems
You should be familiar with the application development life cycle (e.g. agile, iterative or any other model), project management process (e.g. project planning, tracking, control, reporting and so on) and the test process (covering for example, build verification tests, new features tests, automated regression tests, defect life cycle and so on).
Further, you should understand (or learn and understand) the systems used for project management (e.g. Microsoft Project, TargetProcess) and any other systems used for test management (e.g. QC, Bugzilla).

5. Study the application environments
You should at least be familiar with the existing or proposed test environment (a.k.a. QA environment/ test bed/ test lab). It would be good for you to know about the other application environments for example, development environment, and staging environment and the production environment. The information about the application environments is usually available with the people playing the role of release/ deployment engineer or configuration manager or IT support (especially for production environments).

6. Provide feedback
You should provide feedback to the knowledge givers throughout the process of knowledge transfer. It is important to understand and internalize knowledge not only to respect the efforts of the knowledge giver but because this knowledge would be continuously used throughout the project.

Now that you are familiar with the various phases and aspects of the knowledge transfer, you should apply this knowledge in your next project. A good knowledge transfer would give you a definite advantage.