Thursday, February 25, 2010

Testing Games: How to test game software?

Games are available on a number of devices such as computers, mobile handsets, television sets and so on. In this post, I am going to focus on testing the software of a game that runs on a computer. See the video, How to test games? or read on...

How to test games

When you plan to test a software application, you tend to think about the types of testing required. Similarly, when you plan to test gaming software, you should identify the types of testing required. The common types of testing for game software are listed below. Keep in mind that the objectives of a game are quite different from the objectives of a business application. For example, the objectives of a business application could be to provide the given user the features to easily, reliably and accurately perform certain business tasks. In the case of a game, the main objective is having fun. There may be other objectives such as appealing to different types of player demographics, educating the players or developing team skills. Therefore, a game may be required to provide the following:
1. An intriguing or complex plot
2. Realistic/ delightful graphics (including characters, backgrounds and equipment) and sounds
3. Random events (to keep the player interested)
4. Little known facts (to educate the player)
5. Facilitate players to work as teams (In case of multi-player games)
Therefore, the following types of testing should be seen in the light of the given game's objectives.
1. Installation testing
Unless it is a (totally) web-based game, there would be some software that the player needs to install on their computer first. The installation process has a series of steps that should be tested on the common computer configurations (each configuration specifies data like the Processor, RAM, Hard Disk, Display, Display RAM, operating system etc.) used by players.
You should design test cases for the installation steps and then execute these test cases on each of the common configurations.
You should also check the completeness and accuracy of the content in the installation guide. This may be done just once (on the most common computer configuration).

2. Feature testing
This is the most important type of game testing. Typically, the game would have a number of features. You should aim to cover the game's features as exhaustively as feasible. An efficient way to accomplish this is to design test cases in a number of ways, such as:
a. Detailed test cases - These test cases are suitable to cover obvious features e.g. game options, progression from one level to the next, correct working of the controls, start/ suspend/ resume/ stop game etc.
b. Task based test cases - These test cases are at a higher level of detail. They take into account the objectives a player could have e.g. go through each level and win the game, choose and old saved game and continue it etc.
c. Test matrices - These test cases are useful when there are different features for different player statuses e.g. a particular weapon is available to the player only after clearing a particular level, the player can go through an obstacle only when they have collected a particular item before. A test matrix is drawn in the form of a table. The rows could then represent the features and the columns the player statuses (e.g. current level).
Executing the different types of test cases may still not be sufficient to discover a majority of the defects. You should perform exploratory testing. In simple words, this would mean play the game and perform interesting tasks that you think would reveal defects.

3. UI testing
Since games are typically based on user interfaces, this is also an important type of game testing. UI testing includes testing of the graphic elements (e.g. characters in the plot, backgrounds, objects in the foreground) as well as the content (both viewable and audible). An efficient way to test the UI is to first list the checks desired for every graphical element and content type. Then check each item in your list in every logical division of the game (e.g. each screen and each level).

There is an important thing that you should keep in mind regarding UI testing. It is possible that your game is released in multiple languages and cultures. If so, you should create a matrix (a table) listing all the supported languages. Then you should take the help of a speaker of a given supported language to check the content. You should get the content checked in each supported language at least once.

4. Performance testing
A novel story line, incredible graphics and sounds and numerous well-integrated features of a game may still fail to satisfy the player if the game becomes too slow or freezes up. You can check the speeds of operation of the game on common computer configurations. A good way to do this is to identify the common tasks a player is likely to perform. Then you should determine the acceptable times for these tasks. These times are the goals. You should then perform each
task in the game and note the time it actually takes to do so. Since there are a large number of tasks to be timed, you should automate the performance tests.
Another flavor of performance testing comprises testing your game at extremes. For example, run a complete game without any pauses at all or run a game continuously for 24 hours or keep on increasing the number of players if your game is multi-player. Running such tests would give you data on how the performance of your game degrades on high loads. If the game shows performance problems at a realistic load, it means that you have discovered a problem that should be fixed.

You now know about the main types of tests that you can run on a game. Remember that feature tests and UI tests are especially important. Wouldn't you like to use your knowledge to test some game software?

Friday, February 19, 2010

Software Testing Books

Occasionally, people ask me about some good books on Software Testing. I refer them to this list. They say that though these books build up a subject from the beginning, everyone could learn something from them.

Wednesday, February 17, 2010

Knowledge Transfer: What is the best way to do it?

We know that a lot of knowledge is required by the tester before starting any test related tasks (even test planning). The knowledge is gathered and assimilated by the tester before the project starts or just after the project starts. The process by which this knowledge is gained is called Knowledge Transfer. Knowledge transfer is quite visible when a new team member joins the team and the existing team members (including the business people or the client) spend time in communicating with the new member to bring him or her up to speed. In an industry with a fondness for acronyms, knowledge transfer is also called KT.

Knowledge transfer is critical for testing. If your knowledge transfer has gone haywire, you could end up with people questioning your abilities (or worse, you could be out of the project). Remember that your successful performance depends on a good knowledge transfer to you.

Sometimes the knowledge transfer happens in a chaotic, incomplete or hurried way due to any of the following problems:
a. Incorrect identification of knowledge givers
b. Lack of motivation on part of the knowledge givers
c. Communication challenges between you and the knowledge giver
d. Problems due to remoteness
e. Lack of trust (especially damaging to knowledge transfer)

The approach for knowledge transfer should take care of the above problems. Further, the approach should attempt to transfer knowledge that is located obviously in the documentation and processes and not so obviously in the team's behaviors and minds.

The knowledge transfer should have the following phases. Phases 3, 4 and 5 may be done in parallel. Otherwise, it is best to execute the phases in the given order. Even though the knowledge transfer may be initiated and closed by someone else (e.g. the client or the project manager), you are directly impacted by the success of the knowledge transfer. Therefore, you should take an active part in the process.

1. Prepare
The key people possessing the (most) knowledge in the various areas e.g. application functionality, application infrastructure etc. should be identified. It is not sufficient to inform the knowledge givers about the task. The knowledge givers should be motivated to impart the knowledge to you. This is usually best accomplished if the performance of the knowledge giver is somehow linked to the knowledge gained by you.

The one or more mechanisms of knowledge transfer should also be decided:
a. Demonstrations
b. Discussions
c. Providing documents as required by you
d. Replying to questions asked by you
e. Mentoring
f. Instant messaging
g. Wikis

2. Understand the client
It is important for you to understand the client organization (their business objectives, the purposes for which they use the application and so on). If the client cannot impart the knowledge to you, a business person (e.g. the business analyst) may impart the knowledge about the client.

3. Get familiar with the application
This is an important phase of the knowledge transfer since you would be spending a major part of her time with the application. A person very familiar with the application may demonstrate the functions and features of the application to you. Further, you should get a hold of a demo installation of the application in which you may explore the functions and features.
You should also get the (correct versions of) application documentation. Examples of application documentation include the system requirements, functional requirements, installation/ operating manuals, help files/ user guides, test cases, test plans and so on. This is the time to observe the application, read the documentation and ask as many questions are required to learn the nuances of the application.

4. Get familiar with the development, project management and test processes and systems
You should be familiar with the application development life cycle (e.g. agile, iterative or any other model), project management process (e.g. project planning, tracking, control, reporting and so on) and the test process (covering for example, build verification tests, new features tests, automated regression tests, defect life cycle and so on).
Further, you should understand (or learn and understand) the systems used for project management (e.g. Microsoft Project, TargetProcess) and any other systems used for test management (e.g. QC, Bugzilla).

5. Study the application environments
You should at least be familiar with the existing or proposed test environment (a.k.a. QA environment/ test bed/ test lab). It would be good for you to know about the other application environments for example, development environment, and staging environment and the production environment. The information about the application environments is usually available with the people playing the role of release/ deployment engineer or configuration manager or IT support (especially for production environments).

6. Provide feedback
You should provide feedback to the knowledge givers throughout the process of knowledge transfer. It is important to understand and internalize knowledge not only to respect the efforts of the knowledge giver but because this knowledge would be continuously used throughout the project.

Now that you are familiar with the various phases and aspects of the knowledge transfer, you should apply this knowledge in your next project. A good knowledge transfer would give you a definite advantage.

Friday, February 12, 2010

How to select the best automated software testing tool quickly

Well, you know how it is. You have been testing an application for some time now. One fine day, your manager walks over to you. He tells you that the management (or your client) is interested to get the functional tests automated. Since you are very familiar with the application, you should suggest the most suitable automated software testing tool for the purpose. You may know about your application’s technology and about a number of automated testing tools. However, you may not be clear about how to do justice to the evaluation exercise without burning time over this extra work. View the video, Select best automated testing tool or read on.

Monday, February 8, 2010

Responsibilities and Roles of a Software Test Engineer

There are many talented and promising Software Test Engineers in the industry today. Their obvious responsibilities are testing software and reporting defects. However, different companies have different sets of additional responsibilities that are assigned to their respective Software Test Engineer roles. Therefore, I thought it would be a good idea to consolidate a list of the responsibilities of a Software Test Engineer. I hope that this list would help current Software Test Engineers to acquire any skills missing in their skill set. This list would also help any aspiring Software Test Engineers to get an understanding of the responsibilities of this role. View the video, Software tester job role or read on.

Wednesday, February 3, 2010

Software Testing Jobs for freshers: How to get an entry level position?

In several forums, particularly Yahoo Answers, I have seen posts by users looking for entry level positions in the field of software testing. The problem is that most job posts for software testers seek candidates with not only specific skills but also substantial experience (in years) using those skills. This post should help the people freshly out of college and others who wish to begin their career in software testing. View the video, Top tips to get software testing jobs or read on.

First of all, a fresher or a beginner in software testing would do well to ensure that they have the basic knowledge that is going to be needed. In order to achieve this, one might use the following resources:

1. Courses
I have uploaded my entire manual testing tutorials set on YouTube. It is a set of 37 videos that covers all the basic software testing concepts in detail with multiple examples. These videos should be viewed in the order given. These are public videos so you can view them freely as many times as needed to understand them. Click here to see these Manual Testing Tutorials
 
Also, there are numerous institutes that teach software testing. One may be able to find a good local institute. A good institute should offer placement assistance to its students and alumni.

2. Books
There are several good books that teach software testing. Here is one list of such books.

3. Tools
The developers of many popular automated testing tools e.g. QTP, Rational Functional Tester, and LoadRunner (you can learn LoadRunner using my free LoadRunner Tutorials ) make evaluation copies of their tools available free of charge. Several popular testing tools e.g. Selenium (learn about Selenium using my free Selenium tutorials ) , Bugzilla are open source. One may explore such tools to understand their features.

One should use all the relevant job web sites to search the entry level jobs. Some employers may be willing to relax the experience requirement if the candidate has a good knowledge of the testing work. If one is already employed, they may explore the possibility of an internal transfer to a software testing position.

There are a number of jobs posted on software testing forums. Examples of these forums are:



The additional advantage of joining such forums is that one is able to learn from and take advice from senior software testing professionals.

One should take others’ help in gaining the software testing position. In addition to asking for help from one’s friends and acquaintances, one should consider joining social networking sites like LinkedIn (which has an enormous number of professionals as its members). If one is a member of LinkedIn, one has the option available to join several groups which may have their own job discussions. One may also find work on tester communities like uTest.