Sunday, November 6, 2011

Database Normalization: What to test for Second Normal Form?

In the last post, Database Normalization: What to test for First Normal Form?, I explained database normalization briefly. You also saw the tests that should be executed to check the first normal form (1NF). In this post, let us understand the second normal form (2NF) and the tests that need to be executed for checking it. View my video on Second Normal Form with examples. Then read on...

A table that is in 2NF is already in 1NF. Additionally, each column that is not part of any candidate key depends on the entire candidate key (and not a part of it). Now, let us see the examples of some tables that are not in 2NF and how to convert them to 2NF.

Saturday, October 29, 2011

Database Normalization: What to test for First Normal Form?

Designing a relational database to minimize data redundancy (and therefore maximize data integrity) is called normalization. View my video on Normalization and First Normal Form. Then read on...

The concept of data normalization was introduced by Edgar Codd, right in the years after he invented the concept of the relational model of storing data. There are various degrees of normalization 1NF (First Normal Form), 2NF, 3NF and so on. Each degree of normalization is stricter than the previous one e.g. if a table is in 3NF then it is automatically in 1NF and 2NF. In this article, I will explain the First Normal Form and what to test for it. Articles on testing the other normal forms will follow.

Friday, October 28, 2011

Performance Test Scripts Sections

Performance test scripts model the virtual user's expected interaction with the system. A performance test script is usually created within the performance testing tool. The default performance test script generated by the tool needs to be re-factored, parametrized, co-related and unit tested before it can be used in a performance test. Each performance test script contains various sections. It is important to know about these in order to create robust scripts that work correctly.

Saturday, October 22, 2011

SQL Test Online

Structured Query Language (SQL) is the programming language used to find out and modify the data stored in a relational database. Knowledge of designing SQL queries is important in software testing. Take the test below and find out your level of knowledge and the areas in which you should improve. Each question has exactly one correct answer. There is no need to consult any reference for answering these questions.

1. SELECT statement can fetch data from _______ table(s) of the database.
a. exactly one
b. at least two
c. one or two
d. any

2. What values can be included in the SELECT expression (the list following the SELECT keyword)?
a. Any columns
b. All columns
c. Computed values e.g. Price * 10
d. All of the above

3. Which function gives the total number of rows in a table?
a. SUM
d. This has to be done indirectly by executing a SQL query (
e.g. SELECT * FROM Authors) and noticing the number of rows.

4. Which of the following SQL queries is correct?
b. SELECT * FROM Books WHERE Price BETWEEN 10, 25
c. SELECT * FROM Books WHERE Price BETWEEN (10, 25)
d. SELECT * FROM Books WHERE Price >10 AND Price < 25

5. Which JOIN clause returns only the matching values from two tables?

6. Which statement is correct for the GROUP BY clause?
a. GROUP BY allows grouping by only one column
b. GROUP BY needs to precede the WHERE clause
c. An aggregate function needs to be specified based on the column specified in GROUP BY
d. HAVING clause can be used in place of GROUP BY clause

7. What is true about Normalization?
a. It avoids data duplicities within and across tables.
b. It is easier to extend the database structure of a normalized database.
c. A normalized database structure is better than a de-normalized one when the SQL queries against it cannot be predicted in advance.
d. All of the above.

8. Which of these SQL queries is correct?
a. SELECT * FROM Employees ORDER BY LastName + FirstName
b. SELECT * FROM Employees ORDER BY LastName ORDER BY FirstName
c. SELECT FirstName, LastName FROM Employees ORDER BY LastName, FirstName DESCENDING
d. SELECT FirstName FROM Employees ORDER BY LastName, FirstName

9. Which of these statements is incorrect for the UNION operator?
a. Both SELECT statements have the same number of columns.
b. The UNION operator returns values that are duplicated in the two resultsets.
c. The column names returned by the UNION operator are taken from the first SELECT statement.
d. Either of the two SELECT statements can have WHERE, GROUP BY, HAVING and ORDER BY clauses.

10. Which of these is valid for a correlated sub query?
a. It is specified in the WHERE clause of the outer query.
b. It is specified in the FROM clause of the outer query.
c. It uses value in the outer query in its WHERE clause.
d. It is mentioned in the outer query's SELECT clause.

Click the Read More link for the correct answers. 

Sunday, October 9, 2011

Risk Management in Software Testing

Risk management is a critical activity in software test planning and tracking. See my short video, Risk Management in Projects or read on.

It includes the identification, prioritization/analysis and treatment of risks faced by the business. Risk management is performed at various levels, project level, program level, organization level, industry level and even national or international level. In this article, risk management is understood to be done at a project level within the context of software testing. Risks arise from a variety of perspectives like project failure, safety, security, legal liabilities and non-compliances with regulations. An important thing to understand is that risks are potential problems, not yet occurred. A problem that has already occurred is an issue and is treated differently in software test planning. Risk management in software testing consists of the following activities:

Risk Identification
Risks are identified within the scope of the project.  Risks can be identified using a number of resources e.g. project objectives, risk lists of past projects, prior system knowledge, understanding of system usage, understanding of system architecture (see my video, Introduction to Software Architecture)/ design, prior customer bug reports/ complaints, project stakeholders and industry practices. For example, if certain areas of the system are unstable and those areas are being developed further in the current project, it should be listed as a risk.
It is good to document the identified risks in detail so that it stays in project memory and can be clearly communicated to project stakeholders. Usually risk identification is an iterative process. It is important to re-visit the risk list whenever the project objectives change or new business scenarios are identified. As the project proceeds, some new risks appear and some old risks disappear.

Risk Prioritization
It is simpler to prioritize a risk if the risk is understood accurately. Two measures, Risk Impact and Risk Probability, are applied to each risk. Risk Impact is estimated in tangible terms (e.g. dollar value) or on a scale (e.g. 10 to 1 or High to Low). Risk Probability is estimated somewhere between 0 (no probability of occurrence) and 1 (certain to occur) or on a scale (10 to 1 or High to Low).  For each risk, the product of Risk Impact and Risk Probability gives the Risk Magnitude.  Sorting the Risk Magnitude in descending order gives a list in which the risks at the top are the more serious risks and need to be managed closely.
Adding all the Risk Magnitudes gives an overall Risk Index of the project. If the same Risk Prioritization scale is used across projects, it is possible to identify the riskier projects by comparing the Risk Magnitudes.

Risk Treatment
Each risk in the risk list is subject to one or more of the following Risk Treatments.
 a. Risk Avoidance: For example, if there is a risk related to a new component, it is possible to postpone this component to a later release. Risk Avoidance is uncommon because it impacts the project objectives e.g.  delivery of new features.
 b. Risk Transfer: For example, if the risk is insufficient security testing of the system, it may be possible to hire a specialized company to perform the security testing. Risk Transfer takes place when this vendor is held accountable for ample security testing of the system. Risk Transfer increases the project cost.
 c. Risk Mitigation: This is a common risk treatment. The objective of Risk Mitigation is to reduce the Risk Impact or Risk Probability or both. For example, if the testing team is new and does not have prior system  knowledge, a risk mitigation treatment may be to have a knowledgeable team member join the team to train others on-the-fly. Risk Mitigation also increases the project cost.
 d. Risk Acceptance: Any risk not treated by any prior treatments has to be accepted. This happens when there is no viable mitigation available due to reasons such as cost. For example, if the test environment has only  one server, risk acceptance means not building another server. If the existing server crashes, there will be down-time and it will be a real issue in the project.

Few other points are:
1. Risk management brings clarity and focus to the team and other stakeholders. Though the team should avoid burning more time on risk management if it is not providing more value.
2. The risk list should be a live document, consisting of current risks, their prioritization and treatment plans. The test approach and test plan should be synched with the risk list whenever the latter is updated.
3. Bigger projects commonly involve more stakeholders and have more formal risk management process.

Image: jscreationzs /

Tuesday, October 4, 2011

SQL Injection

If you have read my earlier article, Code injection attacks, you would have some idea about SQL injection attack. This post explains SQL injection in detail so that you may understand it well.

What is the SQL injection vulnerability? Vulnerability is a weakness in the application software under test that can be attacked to cause the application (or even the underlying operating system) to behave in an undesirable manner. The SQL injection (SQLi in short) vulnerability lives in the middle-layer or the database layer of the application. It exists when the application executes a dynamic SQL query against the database without validating, escaping or rejecting the unexpected inputs given by the attacker. These inputs become a part of the dynamic SQL query and are executed against the database.

What is the SQL injection attack? It occurs when some text or even another SQL query is inserted into the application's SQL query. Attacks can be successful or unsuccessful depending on the application and the underlying database. A successful SQL injection attack may show confidential data to the attacker, allow the attacker to impersonate another user, increase the attacker's privileges to higher levels, insert/ modify/ delete data in the database tables or even perform administrative operations on the database like shutting down the database instance.

With this background, let us see examples showing SQL injection.

Sunday, October 2, 2011

Team Productivity - 10 ways to ensure that your team members excel and grow

This post is about the softer skills of software test management. It is about how to have your people excel in their jobs. It is something about which I feel strongly. The end result of a project is not the only important thing. Even more important is the career benefit to your team member. Is it possible to run a project such that throughout the project, your team member matures his skills, his attitude and his professionalism? Executing projects consistently like this will ensure that your team member grows professionally. Better performance on subsequent projects will be a given. So, how does a test manager go about having their team members excel and grow? It's not easy. It's also not very difficult. Here is how.

Tuesday, September 27, 2011

Accessibility Testing Checklist

Accessibility is the attribute of a software application that makes it possible for people with disabilities use it. Accessibility is very important because a large number of potential users have limited abilities or disabilities. Examples of disabilities include visual impairments (from colorblindness to partial sight to complete blindness), deafness (partial or complete hearing loss), mobility impairment (inability of use hands or other parts of the body) or neurological (ADD, epilepsy etc.). Examples of limited ability people include people with limited education, old people with medical conditions and children. Thankfully, a number of assistive devices (e.g. screen readers, speech recognition software and Braille terminals) are supported in a number of operating systems. As testers, it becomes our responsibility to ensure that people of limited abilities or with disabilities are able to use the assistive devices with the application that we are testing. So what things we must test to ensure that the software application is accessible?

Sunday, September 25, 2011

A/B Testing

Many websites use a type of software testing called A/B testing or split testing. The objective of this testing is to determine and positively influence the user experience on the website. It involves testing distinct design layout options of the website. A/B testing is also performed on non-web elements of the website such as emails. Many large companies use A/B testing. So, what is A/B testing?

Saturday, September 24, 2011

Code injection attacks

If you are going to do software security testing of applications, you must be aware of possible code injection attacks. This is especially required when testing web applications because they face a hostile environment, the internet. Code injection means adding extra code to an executing application with the purpose of modifying the application behavior. This extra code can be in the form of HTML, Javascript or SQL or even unhandled type of text (e.g. special characters and long strings). Here are the types of code injection attacks.

Thursday, September 22, 2011

Common Security Terms

I thought about mentioning some important computer security terms. It would be good if you know and understand these terms which are commonly used in computer security. See my video, Cyber Security Basic Terms and Concepts or read on.

1. Authentication
It is the process of confirming the genuineness of an entity or data. For example, when a user logs into a website, the website tries to confirm if the user is genuine. That is, he is using the correct user name and password which are active in the site database. There can be other user data involved in authentication e.g. security token, Personal Identification Number (PIN) or finger prints/ retina print.

2. Authorization
It is the grant of permissions to do certain actions. For example on a website, a general user may view their transactions and modify their profile. But, if the general user is inactive, he may only activate his account and not even view his own profile. An admin user may view and modify the settings of the website, but may not view any user's data.

3. Cryptography
It is the use of techniques to assure secure communication. Such techniques include using encryption algorithms and digital signatures. For example, a website may require encrypted data sent from the server to the client browser. This data is decrypted in the client and then rendered to the user.

4. Exploit
An exploit can be a wide range of items e.g. a set of commands, a set of data or even a particular system. It is possible to use an exploit against an insecure information system with the purpose of generating undesirable behavior in the system. Exploits always target a vulnerability (see below) in the system. For example, an attacker places exploit code on a trusted website. A user accesses the site from an insecure client machine which results in the user's browser executing the exploit code. The attacker then runs some attack using the user's machine with user credentials.

5. Firewall
A firewall is a software-based or hardware-based device that can be configured to allow genuine network traffic to go through and stop malicious traffic. For example, a firewall may intercept all network packets sent to an application, such as a browser. If the traffic originates from a known dangerous source, it may drop all the packets preventing harm to the client machine.

6. Identity
It is the set of data that is unique to a person. The digital identity of a person is the identity that is used on-line. For example, the identity of an employee may comprise of attributes such as Employee number, Date of joining, First name and Last name. The employee is then required to use this identity within the organization throughout their stay.

7. Penetration test
It is a test to check the security of a network while pretending to be an external attacker. It is also called a pentest. The goal of a pentest is to gain access and some level of control over a device within the network. Valuable information is discovered by a pentest e.g. vulnerabilities present in the network and the effectiveness of automatic network scanning software.

8. Physical security
It is the presence of physical barriers to information resources of an organization. Examples include gated access manned by security guards, access card swipes required to enter the premises and logged access to restricted areas within the premises.

9. Threat
It is a danger to the information resources of an organization. It can be intentional e.g. an external attacker or accidental e.g. an earthquake. There are usually numerous threats to the information resources e.g. natural catastrophes, power outages, hardware/ software failures and malicious individuals/ organizations. Information resources of different organizations may face the same threats.

10. Vulnerability
It is a weakness within the information system of an organization. For example, software vulnerability arising from insufficient testing, hardware vulnerability arising from its insecure storage and physical site vulnerability arising from its location within a natural disaster prone area.

If you are interested to know about many more computer security terms, you can visit the SANS glossary of terms here.

Sunday, September 11, 2011

Automation Criteria - guidelines on how to write test cases that will be automated

Everyone knows that a strong house can be built on a strong foundation only, never upon a weak one. This post is in continuation to the earlier post, How to write automatable test cases? Test cases here mean "manual" test cases, the kind that a tester can execute against the application under test by hand. Each of the following guidelines is also applicable to create valid test cases that would be executed, so there is no special effort here to prepare such test cases for automation.

1. The test case must be correct. It is obvious that the test case must be correct with respect to the workflow and expected application behavior. This guideline is especially important for automation because though it is possible for a manual tester to ignore obvious incorrectness in a test case, the automated test script would not be able to do the same. False negatives would be reported every time the automated test script is executed.

2. All the business scenarios must be covered by the test data. This refers to the completeness of the test case. The test case must contain test data for all applicable business scenarios that users would face.

3. The test case should have sufficient detail so it is possible for another person to execute it without asking questions or getting clarifications. Pre-conditions, test steps, test data, expected results and post-conditions are important components of a test case. The test case should be written with the target consumer in mind. If the automation engineer has good knowledge of the application under test, the test case components may be summarized. If not, they should be detailed out.

4. [Important] The test case must be capable of finding bugs in the current release of the application. If a test case has not caught a bug in the last few releases of the application, the likelihood of it doing so now is limited. Extra effort is required to automate test cases. So, why not automate the test cases with a high likelihood of catching bus?

5. The test case should have been accepted by the team. The test case that would be automated should not be in a raw state e.g. just whipped up by someone. It should have been reviewed/ discussed, stored in the correct file and accepted by the team as a valid test case.

6. The test case should be under version control. Placing the test case in the version control repository shows the changes made to it subsequently. Changes to the test case should be propagated to the automated test script, whether the latter is under construction or already built. Therefore, there must be a process to update the automated test script whenever the test case is revised and accepted.

Correct, complete and up-to-date test cases are important assets for any testing team. Due attention is paid to such test cases. Similar attention should also be accorded to the automated test script of such a test case. After all, its the same test case, just written in a different format. Therefore, the automated test script should be reviewed/ discussed, accepted and placed under the version control repository. The results of each execution of the automated test script should be given similar attention.

Saturday, September 10, 2011

Do managers have bigger brains?

Within the last couple of days, I have been intrigued by a news item that said that Managers have bigger brains. Then, I used Google to find out the related text. The purpose of this post is not to analyze whether or not "managing" results in a bigger brain, or even the implications of this discovery. The focus of this post is to analyze how simply using natural language to describe the result of a study can distort what is being communicated. Then, move on to the application of this analysis to requirements analysis in software testing.

Managers have bigger brains: Since this study was conducted University of New South Wales researchers, I searched the UNSW website and found this article here. My comments were:
1. I found the following text at this link "UNSW researchers have, for the first time, identified a clear link between managerial experience throughout a person’s working life and the integrity and larger size of an individual’s hippocampus – the area of the brain responsible for learning and memory – at the age of 80." So, they are not talking about the entire human brain but only one of its parts.
2. The article talks about finding a relationship between the size of the hippocampus and the number of employees managed. It does not state the exact relationship.
3. Per the article, the researchers used MRI in a sample of 75 to 92 years old. Around the middle, the article moves on to the relationship between exercise and learning and other topics presented in the symposium where this study was presented.

This news item also appeared on other websites such as MSN India here.
4. I found the text "Staffers agree or not, managers do have bigger brains, says a new study." The prior article had no mention of the staff agreement to the manager having a bigger brain. So, did the research take the subjects' staff's agreement into account?
5. This news item says "Researchers, led by the University of New South Wales, have, in fact, for the first time, identified a clear link...". The previous article just mentions "UNSW researchers". So, were there teams from elsewhere involved in the research?

What can we take away from the analysis?
a. It is possible for people to over-generalize or over-specialize a description. So, we should probe further to find out the caveats and exceptional conditions. For example, a requirement may say "The user account shall be locked after 3 unsuccessful attempts". On probing further, we may find that this is true for all users, except the administrator user. The system may be rendered in-operational if the administrator gets locked out :)
b. It is possible for people to just provide references to other information, without naming it explicitly. Instead of relying on our assumptions, we should ask for the reference directly. For example, a requirement may say "Other requirements implemented in the previous release shall continue to be supported". We should ask for which release - the last major release, the last release (major or minor), the first release or any other release? If there is a conflict between the requirements of this release and the "previous release", is it okay to have the requirement of this release take priority?
c. We should question the source of the requirement. Has it come directly from the customer or system analyst or just a suggestion from somebody? In other words, is it a genuine requirement? For example, someone may suggest making reports completely configurable. It may be against the client's objective that any user comes up with any format of a report, leading to confusion. The suggestion should be declined. Of course, suggestions made by the management need to be declined tactfully, if infeasible to implement.

Saturday, September 3, 2011

Testing Training - How I train professionals?

Ever since I started my career in software testing, I have been asked to give training sessions on testing topics. The topics on which I have trained people include basics of software testing, writing correct and re-usable test cases, rapid test execution, bug tracking systems and defect management to test automation approaches, performance testing, system security testing and test methodologies. 

I do have a couple of advantages. First, two years experience in giving software training during my initial career. Second, I have always been keenly interested in how people learn (probably because of 6 teachers in my family, including my mother). I am life-long student of the psychology of learning.
Let me explain the approach I use during training. This is both for your benefit and my benefit. Your may benefit from these tips by increased recall of the participants after the session which leads to more application of the training material in projects. I will benefit by looking up my tips to ensure that I continue to follow them and build them further.

1. Know your topic
This is most important. You should know your topic really well. Not just a little more than the participants, but many times more. Why? In order for learning to take place, the participant has to believe in the superior knowledge of the trainer. For example, let us say that you are training on creating automated test scripts. You have done this before using a functional test automation tool. Someone in your class asks about parameterization and you are stuck. What will happen? What you say from that point onwards would not be credible. Therefore, give training only on topics that you know inside out.

2. Plan your training session
You should know the objectives that the training session should achieve. This information is available from the sponsor of the training. Also, if you know the participants, you can find out their current knowledge level and their expectations from the session. If you don't know the participants in advance, you should spend some time at the beginning of the training session to find this information. For example, if you are training on system security, the participants need to know the basic security concepts like information integrity, information confidentiality and information availability. The training session should cover the material required to increase the participants knowledge from the current state to the desired state.

3. Create your training material
Once you are clear on the objectives of the training, the next step is to design the sub-topics and training material. For example, if you would be training on writing test cases, you should cover the inputs to the test cases (requirements documentation, design documentation, prior test cases, test case formats etc.) and tips on how to write test cases (with all scenarios, pre-requisites, test steps and expected results). The training material can include practical examples of good test cases.
One caveat: The training material should not be too lengthy or cluttered. The main focus should be on the participants understanding the topic well. They can always look up the references if they need more details later.

4. Starting the training session
For learning to take place, the participants should be engaged in the training session. A useful way to do this is to explain the objectives of the training session in terms of what they already know and how the training will help them work in a better way. For example, if training on bug tracking systems, the objective is to use the bug tracking system more efficiently, you could mention that you will share tips on using the bug fields correctly and setting up email notifications for more efficiency.

5. Treat the topic logically
If you plan your training session well, you would have identified the logical sequence of the sub-topics. Answer questions from participants as required but stay on course. For example, if training on performance testing, the logical sequence for you may be creating the automated test scripts, parameterization and correlation, modeling the test, test execution and analyzing the results. If you have moved to test modeling and you get a question on parameterization, answer it quickly and then say you are moving back to test modeling. Also, explain any new concept or term with more than one example as soon as you introduce it.

6. Make the session interactive
People like sessions during which they can ask a question any time. If the question is related to the training session, you should answer it. In fact, you should always pause for questions on the conclusion of a sub-topic. If a question is not directly related, you could park it. For example, if you are training on functional test automation approach and someone asks you which tool is best for automated testing, you could say that it is not directly related and you would take it offline later. You always need to be respectful of the participants' time that they have chosen to spend listening to you.
One trick I use to keep my session interactive is to pretend that a term or word has slipped my mind. This forces people to think and they come back with suggestions energetically.

7. Summarize the training session
After completing the session and answering questions, be sure to summarize the main points in the training session. The material covered must be linked to the objectives of the training session. For example, if you trained others on object repositories, you could say that by now the participants should have awareness of the types of object repositories and they should be able to make an informed decision to use the best type in their project.

8. Finishing up
Spend a little more time sharing the training material, providing further references and providing a contact to answer questions later on. Also, reiterate the expectation from participants now they attended the training session.

Well, these were but some of the guidelines I use when training others. Your training session should be more productive if you use the above. What do you think is most important in making a training session successful?

Sunday, August 21, 2011

Database Testing Example

I have explained database testing with examples in my video, Database Testing and SQL Tutorial for beginners.

Now, I received several emails from the my viewers asking for advice on how to practice database testing. This post contains one example to practice the same. I have explained SQL queries and SQL joins in my SQL Tutorial for Beginners | SQL Queries Tutorial using multiple examples.

Create a practice database with three tables, Products, Orders and OrderDetails or use the database that I showed in my SQL Queries tutorial above. It does not matter which database management software you use. It is simple to anticipate the relationships here. Products table is related to OrderDetails table, via the ProductId field. Orders table is related to OrderDetails table via the OrderId field.

Now, write and execute queries to check the following:
1. Is there any product which exists in the OrderDetails table but is missing in the Products table?
2. Does any OrderDetails row have an OrderId that is missing in the Orders table?
3. Is any ProductName, Serialnumber or ProductDescription duplicated in the Products table?
4. Does any product have a negative UnitsInStock?
5. Does any order have a ShipDate which is earlier than the OrderDate?
6. Does any order have a future OrderDate?
7. Does any OrderDetails row have a Quantity value less than 1? Or a Quantity value which is fractional?
8. Does any OrderDetails row have a zero or negative SalePrice or UnitPrice value?
9. Does any OrderDetails row have a UnitPrice for a product, different from that product's UnitPrice in the Products table?
10. Can you enter Price values (UnitPrice in Products table and SalePrice and UnitPrice in OrderDetails table) in decimal (up to 2 digits after the decimal point)?

Saturday, August 20, 2011

How to impress your boss and team?

Would it not be great if you could impress your manager and team by putting in just a little more effort? Here are some ideas for building a great perception of you. Note that 3 out of the 8 ideas relate to communication, very important to impress others.

1. Maintain stable work timings
It is important for others to know when to reach you. You can make it easier for them if you are always in office, say before 9 a.m.. Maintaining fixed in-timings shows commitment to work. When you are on vacation, make sure that you set up an Out of Office message notification with information of another person to contact, if urgent.

2. Ask lots of questions during meetings
If you are attending a meeting, you might as well ask questions. Use the 5 W's (What, Why, Where, Who, When) and How to frame questions to get more information for you and others. Asking questions during meetings shows that you are engaged with the topic and eager to understand more.

3. Volunteer for high-visibility assignments
Volunteers are asked for in meetings or written communications. Be eager to take up a task that only one person can do, but which will benefit the whole team. Also, look for small tasks that nobody else is doing. And complete them.

4. Communicate regularly
Whatever you have completed and are doing at present, be sure to communicate it regularly. If you maintain data about your daily tasks, it will be possible for you to communicate daily. Then summarize weekly, monthly and even annually, and communicate those summaries also.

5. Communicate using the correct vocabulary
Different companies and teams have different preferred vocabulary. If you use the specific terms preferred by your manager and team, your communication will be easier to understand. Also, these words show that you are engaged with your team and company.

6. Keep building material for later communication
Have you seen well-structured emails from others with a number of original thoughts? And wondered how much time and effort they spent on creating that email? You, too, can do the same. For important emails, instead of writing whatever comes to your mind and shooting off the email, collect your thoughts in the draft first. Don't send it. Keep updating the draft when you get any good ideas. Soon enough, you will have a worthy email that will impress others.

7. Ask for inputs
When you create a project deliverable (test plans, test cases, automated test scripts etc.), always ask for review comments from your manager and team. You may not get review comments every time. But, whenever you do, it is a good input to refine your deliverable. Also, this shows your commitment to quality.

8. Say "Hi"
Finally, take a couple of seconds to wish your colleagues whenever you meet them. This helps others think of you as a courteous and approachable person.

Please let me know if you liked these ideas and the benefit you got from them.
Image Courtesy: renjith krishnan /

Friday, August 12, 2011

Why bad software testing happens?

It is possible for software testing to go wrong in a project. Look out for these red flags in your project and make corrections immediately to avoid bad testing.

1. Customer inputs ignored
2. Poor knowledge of the application under test
3. Application technology ignored
4. Incorrect test strategy
5. Old test process followed mechanically
6. Unorganized testing
7. Poor observation and analyses
8. Insufficient communication
9. Lack of perseverance
10. No self-critique/ correction in the team

Monday, July 18, 2011

Automated test script review checklist

Reviews are a great way to find common problems with a number of test automation artifacts, such as automated test scripts like the ones that I wrote in my Selenium Python tutorials. By an automated test script, I mean script code implementing a test case's test steps and expected results e.g. the second script in my Selenium Python Tutorial 2. The automated test script may be written in any programming language or scripting language. It may be generated by a functional test automation tool or written by hand.

Here is a checklist that you can use to review your or others automated test scripts. You should customize the questions according to your specific needs. For practice, review the automated test script to answer a quiz automatically. Post your review comments below.

1. Does the script implement each step of the test case?
2. Does the script implement each step in the correct order?
3. Does the script implement each validation given in the test case?
4. Does the script have each validation after the correct step?

5. Does the script logic call the correct functions? Are these functions called with the correct arguments?
6. Does the script take application input values from the correct input test data file(s)?
7. Does the script execute user operations against the correct objects?
8. Does the script give the application sufficient time to transition to the correct test state e.g. the correct screen?
9. Does the script validate application output values against the correct output test data file(s)?
10. Does the script validate the state of the correct objects?
11. Does the script load only the necessary objects in memory?
12. Is the script free from unnecessary delays?
13. Does the script handle application errors as designed?

14. Does the script have a unique identifier?
15. Is the script free of any magic numbers or magic strings?
16. Is the script commented at each appropriate place in it? Is each comment correct?
17. Is the script free from unnecessary or extra code?

Let me know if you find this checklist useful in your reviews.

Saturday, May 21, 2011

Automate tasks in software testing

When the term automation is mentioned, it is common for people to think of automated tests. But, time and effort can also be saved by automating other tasks such as generating test data and reporting test results automatically. A few factors should be considered for effectively deciding whether to automate a task or not. Last year, we had fruitful discussions on this topic within the STS group.

Let us see some candidate tasks for automation. Testers spend a lot of time on these tasks. If such tasks are fully or partially automated, the saved time can be spent on testing the system.

1. Test identification based on risk or other factors
List all the available test cases along with related features, components and priorities. The automation can take the inputs e.g. impacted features and generate the list of test cases that should be executed on the new application build.

2. Test estimation
Automation can be used to help estimate the test effort and duration. This can be done based on the preferred estimation approach. Whether by querying the historical data for actual efforts and durations, or by applying a custom formula to estimate test effort and test duration.

3. Generation of test data
Build a library of business rules for your test data. Build the initial seed data manually. The automation can use the seed data and generate test data based on the chosen business rules. Since test data generation is usually consumes much time, the automated generation can be performed ahead of time. Then, the pre-generated test data can be used directly during test execution.

4. Generation of bug reports
This automation can be built into the test automation framework. Whenever an automated test script confirms an error, it logs into the bug tracking system and reports a bug with required information. Such as bug title, steps to reproduce, test data used, environment used and so on.

5. Generation of test reports
This automation can execute queries against the test management system (and bug tracking system, if different) and generate test reports. Even distribute them by email or publishing to a website.

6. Release notes preparation
Release notes contain both static and dynamic data. This automation can execute queries on the test management system to retrieve the dynamic data such as features passed, bug fixes passed and known bugs.

Before prioritizing the automation development, always analyze the following factors.
a. Degree of automation achievable
b. Skills required to automate
c. Effort required to automate
d. Number of proposed users
e. Effort required to train users
f. Effort required to execute automation
g. Effort required to maintain automation
h. [Important] Manual effort saved

In the future, I see a number of such tasks automated with the help of vendor tools or bespoke in-house automation.
Let me know if you liked this post. I would love to know your thoughts on this topic.

Sunday, May 15, 2011

Software testing myths

Myths abound in the software development industry. Software testing is no exception. People tend to hear such statements and pass it on to others minus the full context. Not only that, they even tend to use such myths when making own decisions, especially when in hurry.

1. A ratio of 1 tester to 5 developers is enough to get good test coverage.
This myth may be true in long-term new development projects with multiple stages of testing (e.g. SIT, alpha, beta and UAT). This ratio may fall short in:
a. Maintenance projects with highly inter-dependent components, requiring testing of all impacted components
b. Projects requiring huge effort to create an independent test environment
c. Projects requiring extensive regression tests (due to, say, business impact, contracts or regulations)

2. Anyone can test software.
Anyone can test software if they know what to do. A professional tester is well-versed in the project requirements, software testing concepts, test design and test execution techniques and is also an assertive communicator.

3. Developers can test software better than testers. After all, they are the ones who developed it.
While developers know the code they develop intimately, they may not be the best people to test it (or test code written by other developers), especially at the integration level or system level. Testers execute their tests in a clean test environment with a fresh mindset, doing both positive and negative testing. Developer testing complements (and does not substitute) testers' testing.

4. Test cases are only derived from requirements.
Requirements are only one input to write test cases. A professional tester uses many other inputs like design documents, standards, checklists, prior test cases, past bugs reported by the customers and prior versions of the application/ similar applications.

5. Testers need only execute a set of test cases.
A static test case suite is unlikely to lead to the discovery of new bugs in the application. Before execution, the test suite should be updated by removing the useless or redundant test cases, completing or correcting the existing test cases and adding new test cases to increase the likelihood of discovering fresh and interesting bugs in the application.

6. Tests already executed need not be repeated in the release.
It depends. If the new builds do not impact the application areas covered by the tests already executed. Much more likely, some tests need to be re-executed to re-gain confidence in those areas.

7. Testers test every permutation and combination of inputs a user can provide to the application.
This is not possible except in very trivial cases. Even testing all possible inputs of a form with a few text, date and numeric input controls within the given time may well be impossible. What testers actually do is to design the minimum number of input sets that tests each input control. Specific inputs to each control are limited by using testing concepts like equivalence partitioning.

8. Testers can test equally well (if not better) when the time is short.
Nobody can perform a thorough job if rushed. Testers are no exception. If the available time is reduced too much, a tester prioritizes the tests to execute and keeps executing the higher priority tests until time runs out.

Image: Idea go /

Monday, May 2, 2011

Simple testing

Lately (actually since a while), I have been looking for ways to increase my productivity. One of the ways I recently came across is simplification. Why?

1. Better focus, especially on creative or complex tasks
2. Less stress, due to elimination of so many unimportant tasks
3. More time on hand

So, how to simplify our software testing tasks? Here are some examples. Design your own according to your unique situation.
1. Analyze your work and establish what is important in your role. Put it down in a sentence. For example, Maintaining your test suite, executing it and reporting defects may be critical to your role. Formatting your status report or pointless gossiping with your colleagues is definitely not.
2. Allow yourself time to plan. Plan simple i.e. think about the easiest set of actions that will complete your task.
3. Guard your time. If someone interrupts you, dismiss the interruption asap. If someone approaches you for help, consider helping once you are done with your tasks or say no.
4. Own your work. If some task had a problem and requires re-work, don't blame other people. They would respond with their reasoning and you would end up spending time on this exchange. Then attend to the problem. Instead, just fix the problem at once and move on.
5. Use software apps to your advantage. See examples.
a. Block your mail calendar for important tasks in advance and in order of importance. This would allow you to see your upcoming tasks for the next day, week or month and you can better prepare for them.
b. To be responsive, check your email a limited number of times in the day. If you leave it on, you would get distracted every time you get a new email. And you will have the urge to stop your current task and read/ take action on the email you got.

This was a short and simple post and it made me feel better writing it.

Thursday, April 28, 2011

Interviews of Thought Leaders - Freddy Gustavsson

Having started his career as a software developer, my dear friend and a key member of the Software Testing Space group, Freddy Gustavsson made the leap into testing in 2001 after realizing that a carefully designed test approach will greatly increase the chance of success in any software project. He is interested in all parts of the test process, and has experience from several international projects. He works as a consultant for System Verification in Gothenburg, Sweden, where he specializes in test strategy, test design, test automation, test process improvement and test education. Freddy is an ISEB/ISTQB certified test analyst who also teaches courses on software testing and serves as a member of the internal Senior Advisory Board.

Here is the interview. Enjoy and learn.

[Inder] Did you start your career with software testing or have you held other software roles as well?
[Freddy] I actually made somewhat of a detour to reach my current position in software testing. At first I had no intention at all to work in the IT industry. Instead, I wanted to pursue a career as a teacher of languages at college level. So I studied German linguistics for two years at the university and prepared for some advanced English studies. However, my passion for web development grew stronger, and I decided to make a shift. In 1999 I started working as a web developer for one of Germany's e-business providers. There I learned tons of good stuff about development, projects, team work and testing. In 2001 I joined the QA team and became a tester. Later, I've finished a 3 year university education and a B.Sc. in Software Engineering because I wanted to understand my profession truly. I've also taken complementary courses in project management, usability, accessibility etc. Additionally, I spent one year teaching programming courses at university level, which was useful.

[Inder] Not all people appreciate software testing as a highly intellectual work; they feel that it is quite easy to test. Can you talk about few technical challenges that you have faced in your career?
[Freddy] It's a known fact that some employers will view testing as an entry point to development. It's as if they told their candidates: "We're sorry, but you're not qualified to play with the developers yet. But let's move you into testing. If you work hard and prove yourself worthy, then some day you might actually be allowed to work in development." Even worse is the situation where employees, who were rejected by other disciplines, are brought into testing, because managers feel that they "do least harm" there. This approach is just so wrong.
Testing is by no means a trivial task. Anyone who has worked as a professional tester knows this. However, not all people coming from other disciplines will understand testing well enough to realize its challenges. They might think of testing as "happy testing" or "randomly clicking around", not realizing that testing requires a controlled process. The presence of structure is vital to our job. At the core of the tester's job is the comparison of actual and expected results. However, this is surrounded by numerous items such as plans, strategies, procedures, methods and tools. Designing and implementing all of this in an effective and efficient way requires excellent business, technical, administrative and social skills.
Common technical challenges include quickly learning new applications and tools, understanding the environment in which each tool operates. Sometimes you also need to learn new protocols, scripting languages, or get an understanding of complex system architectures. This is part of the tester's job. Being curious and eager to learn new things is definitely helpful.

[Inder] Looking back at your software testing career, what have been some of the highlights?
[Freddy] The first highlight would be the recognition of testing as a profession. The second would be the insight that some of my carefully crafted test cases were actually detecting failures. The developers would lovingly refer to me as The Merciless Tester. Their indignation over each found problem would not last long. Instead they asked me to do more testing on their code. Although I have occasionally, and unintentionally, upset a few people through my work, in most cases the work with other disciplines has been both interesting and rewarding. The third highlight would be the move into consulting and a focus on test strategy, which might be the best job ever. Number four is the mentoring and educative role where I get to spread the word and (hopefully) make other people interested in learning more about our craft. Highlight number five would doubtless be this interview. ;-)

[Inder] Do you find testing enjoyable? How can software testing professionals derive more satisfaction from their craft?
[Freddy] Yes. Just like for other professionals there are a number of things that motivate me to do a great job. The most important is probably the feeling of making a valuable contribution to the client. Sometimes also to the community. For instance, a few years ago I was on the system test team for the national Swedish command and control system. The system was built for operation in emergency situations where ambulance, police or fire brigades might be needed. Any failures in the software might possibly have catastrophic (life or death) consequences. In that case every defect found and removed would benefit the whole community. As a tester you knew your work made a difference.
Having a good, respectful working relationship with coworkers and managers also makes it to the top of my list. Another important thing is the possibility to control your job, to be able to suggest ideas and improvements.

[Inder] What are some good resources for people wanting to enter software testing or establish them in this career? What activities can one do in their spare time to enhance their testing skills?
[Freddy] I want to refer to a good seminar on this topic, which I attended at the EuroSTAR 2010 conference in Copenhagen. Markus Gärtner from Germany gave a presentation on alternative paths for self-education in software testing. A number of options were presented:
  • Using social media (e.g. Twitter, LinkedIn, web sites, forums)
  • Learning to program (e.g. scripting languages, design patterns)
  • Reading books on testing
  • Joining online testing courses
  • Participating in testing challenges
  • Participating in organized problem solving activities like testing dojos, weekend testing or the Miagi-Do school of testing.

[Inder] What are your future career plans?
[Freddy] I look forward to an exciting future in which software testing will undoubtedly play an important role. Personally, while keeping an eye at the entire field of testing, I plan to specialize further into the areas that interest me most: test strategies, test process improvement and test design methodologies. I also plan to extend my teaching assignments in the future, since lecturing is a great way to spread the knowledge about testing while learning a lot myself. As they say: you learn as long as you teach.

Saturday, April 23, 2011

Team productivity - How not to bring it down?

If you are fortunate enough to lead or manage a team of software test engineers, consider it a valuable responsibility. Check yourself if you find yourself adopting any of the following tactics. If you do, you can be assured that your team would be working far below their potential. And that would not the only problem you face. I am going to refer to the hypothetical Lead or Manager as LM for my examples below.

1. Poor communication
The LM knows about the incoming project or test run. She has been involved in planning, meetings with other stakeholders and knows a lot about it. But, the quantity of the information passed to the team members is so low or the quality so poor, that they get stuck at every step. They make assumptions and their work de-rails. Unless, of course, they check with her at each step. Takes up their time away from project work.
A variant of poor communication is an overdose of information. The LM floods the team members with documents not long before the start of the project. Worse, the info flood takes place just as the project starts. Now, a team member has to make one of two poor choices - stick to the schedule unprepared OR prepare and lag behind schedule.

2. Stalling
The LM is too process-oriented. She just loves checklists. A team member can consider himself successful only if she completes each of the 1,000 tasks in the checklist successfully. Did I mention that this is in addition to his testing task?

3. Unnatural competition
The LM promotes competition among the team members. Who writes test cases fastest? Who creates bug reports that developers always agree too? Who provides the LM the test results in the exact format that she likes? All this means that not only each team member has to do his work; he also needs to watch what others are doing. Nobody wants to be the last one, so the team members take short-cuts. Project work suffers.

4. Back-stabbing
The LM talks sweetly. After receiving so much appreciation (verbal and written - but always one-to-one) from her, a team member can never suspect the true feelings the LM has bottled up inside. These feelings are released in the reports the LM gives to her manager, other managers, customers and HR. The team member does come to know about the real feedback provided by the LM, but only when the damage is already done.

5. Feedback given unequally
Her praise is concise and in private. Reprimands are public affairs listing the mistakes committed in the past, the current lapse and hopelessness for the future. Needless to say, such reprimands make the team member quite uncomfortable and unsure of how to see his co-workers in the eye again.
A variant is when the LM provides no feedback. The team member has no idea whether he is performing well or poorly. The team member does not know what is round the corner. He does not know what he would be doing tomorrow.

6. False promises
The LM says anything to get the work done. It works, but only the first few times. In time, the team member is disillusioned.

7. Taking undue credit
The team members take the difficult project as a challenge. They work professionally and very hard. They surmount the obstacles. Even though tired, they don't cut corners. The LM takes all the credit for her "leadership" in time of need.

8. Scape-goating
The LM schemes. She already knows the personalities of her team members. She creates a Plan B (and Plan C) if things go south. When they do, she lets any one problem develop, gathers substantial "data" incriminating a chosen team member and dumps the entire situation on his head.

9. Unwillingness to change
The LM is aware of the negative effects some of her actions have on the team. But, it has worked for her in the past and she sees no incentive to change. So, if he is able to work with her, the team member makes the necessary mental adjustments or start looking elsewhere, in time.

Final words - As human beings, we tend to make the best possible choice. As I mentioned above, if you lead or manage a team, you have a big responsibility. Keep in mind that your actions affect not only yourself but also your team members. Take the correct action, even if causes pain in the short-term and is difficult. It will benefit both you and your team.

Thursday, April 14, 2011

What is software regression?

Things are not as good as they used to be.

Before one can do an informed regression testing, it is important to understand software regression, which can happen after an event that changes the system. Software regression is deterioration in the software. Such decay can be functional, meaning one or more functions working earlier no longer do so. Or, it can be non-functional, for example, the software becomes slower/ outputs less or becomes (more) vulnerable to security threats.

Monday, April 11, 2011

How to do end to end exhaustive testing?

Testing a software application (except maybe a very simple program a few lines long) may well be an impossible task due to large number of:

1. All possible inputs
2. All possible input validations
3. All possible logic paths within the application
4. All possible outputs
5. All possible sequences of operations
6. All possible sequences of workflows
7. All possible speeds of execution
And the above for just with a single user
8. All combinations of types of users
9. All possible number of users
10. All possible lengths of time each user may operate the application
And so on (we have not even touched the types of test environments on which the tests could be run).

However, it is possible to exhaustively execute your test suite using the following tips:

1. Your test suite should have test cases covering each documented requirement. Here my assumption is that each requirement is documented clearly.
2. The test cases should be specific, concise and efficient. Each test case should have clear and unambiguous steps and expected results.
3. The configuration data, input test data and output test data should be clearly specified.
4. You should have a clean and stable test environment in which to execute your test suite.
5. In a perfectly working application, it should be possible to execute each test case in the suite.
6.. Each confirmed bug (found during testing or found by the client) should result in another test case written or an existing test case updated.
7. Important: You should not assume the correctness and completeness of your test suite by yourself. Review of such test suite by peers, business people, managers, clients and users may provide you valuable inputs to correct your test suite.
8. Discipline in maintaining your test suite and executing it would go a long way in preventing bugs leaked to the clients/ users of your application.

Saturday, April 9, 2011

Conventional wisdom applied to software testing

Here are the conventional English proverbs as I see applied to the field of software testing:

1. A journey of a 1000 miles begins with a single step.
A test run of a 1000 test cases begins with a sanity test.

2. Genius is 10% inspiration and 90% perspiration.
Software testing is 10% test preparation and 90% test execution.

3. As you make your bed, so you must lie in it.
As you write your test cases (good or poor; complete or incomplete), so you must execute them.

4. A bird in hand is worth two in the bush.
A confirmed bug is worth two suspects.

5. A chain is no stronger than its weakest link.
The quality of an application is no better than that of its most buggy component.

6. Discretion is the better part of valor.
Review is the better part of executing a task.

7. Empty vessels make the loudest noise.
People who least understand software testing speak of it in the most black-and-white terms.

8. An ounce of prevention is worth a pound of cure.
One bug fixed in the beginning is worth 16 updates.

9. A penny saved is a penny gained.
One test case re-used is a test case written.

10. Where there's a will, there's a way.
Where there's an application, there are bugs yet to be found.

Hope you enjoyed these :). If you like any other English saying, let me know. I will translate it to software testing.

Sunday, April 3, 2011

Checklist for application release notes

It is a norm to distribute release notes along with a software release. Software release notes are similar to the product literature that you get when you buy a physical product. You may vary the contents depending on whether the software release is a major release or a minor release, an upgrade to the application or just an update, customized for a specific customer or available generally. Find answers to the following questions to test the release notes that you receive before software testing.

Main information
  1. Does it mention the application name and the correct version number?
  2. Does it correctly list the features released and/ or bugs fixed in the software?
  3. Does it mention the correct software vendor name and date of the release?
  4. Does it mention the release numbers it requires as prerequisite or the release numbers it supersedes?
  5. Does it mention each of the available distribution modes (e.g. website download, email and disc)?
  6. Does it mention the correct distributed media (e.g. install package, executable programs or scripts) in which the software release exists?

Release feature information
  1. Does it explain each new feature (summary, benefits and details of how it works)?
  2. Does it explain bug fixed in this release?
Supporting information
  1. Does it list the minimum and recommended hardware, software and network requirements to deploy and use the release?
  2. Does it provide steps to backup the existing configuration and user data in the system?
  3. Does it provide detailed steps to install the release?
  4. Does it provide steps to restore the configuration and user data of the previous release?
  5. Does it provide the contact channels in case the software does not install or installs with problems?
  1. Is the release notes available in a file format that can be viewed by the customer?
  2. Is the file format consistent with prior releases?

Sunday, March 13, 2011

How to identify high performers in your team?

Having worked with several high performing software testing professionals, I find that it boils down to two things. A high skill level and a positive attitude.

And both are required. A high expertise but no willingness to apply it in the project would result in meager performance at best. And tons of positive attitude but a low skill level would also not translate to solid work. Having said that, attitude is the more important of the two. Why? Skill level can be increased by putting in time and effort but it is much more difficult to change a poor attitude.

I have found consistent high performers to exhibit the following four characteristics:

1. Demonstrates a higher technical knowledge
This person knows more. And it is evident in a number of ways, the things this person says or does, the questions he asks or the responses he gives.
One day we were having a team meeting. I brought up the topic of pair-wise testing to discuss whether it would be useful in the project. To my pleasant surprise, a high performer of our team already knew about this test design technique. He ably explained this technique to the team and we ended up identifying test items that would benefit from pair-wise testing.

2. Communicates more
High performers communicate more than other team members. You can see them asking more (insightful) questions when new work is being explained. They also keep others abreast of their current status and any current/ foreseen problems.
I fondly recall one high performing team member. Whenever I assigned him some work, I would definitely get a response email. The email would mention any unresolved questions and his schedule for this work. Then the work would be completed per schedule and I would get another email summarizing what had been done, the approach used and next action items.

3. Delivers more or higher quality work
By definition, high performers deliver better performance. You would find them doing one or more below consistently:
 a. Finishing tasks ahead of schedule
 b. Finishing more tasks than initially planned for them
 c. Finishing tasks with quality better than anticipated

4. Loves challenges
All high performers love a challenge. In fact, lack of challenges may be boring or even de-motivating for them. So, it is important to raise the bar for them from time to time.
One new addition to our team faced a series of challenges. The industry domain was totally new to this person. A couple of months later, he was using the same vocabulary as the rest of us. He came from a manual testing background. He had to work on the automated testing project. Again a few weeks later, he was very hands-on with the automation tool, scripting business workflows with ease, correlating automated test scripts and even able to perform a reasonable analysis of the test results. Every time he faced a new challenge, he came on top of it.

Now you know about the characteristics of high performing software test engineers. These qualities are not different from what one may expect from high performers in any other field. Think about each of your team members in light of the above four qualities. You would know who are the high performers are. You can also develop the above qualities to increase your own performance. Good luck.

Saturday, March 5, 2011

How much time should a tester spend testing every day?

Not so long ago, I went to attend a walk-in interview. While we were all waiting to be called, I struck a conversation with a test engineer who had also come to be interviewed. One of the things we talked about was the daily routine of this person. Here is a rough daily schedule:

9:00 a.m. Arrive at office and settle down
9:15 a.m. to 9:45 a.m. Check emails
10:00 a.m. to 11:00 a.m. Team meeting
11:00 a.m. to 1:00 p.m. Test the application (2 hours)

1:00 p.m. to 2:00 p.m. Lunch break

2:00 pm. to 3:30 p.m. Test the application (1.5 hours)
3:30 p.m. to 4:00 p.m. Log bug reports (0.5 hours)
4:00 p.m. to 5:00 p.m. Enter test results in the test management system and enter timesheet
5:00 p.m. to 5:30 p.m. Reply to questions on prior bug reports
5:30 p.m. to 6:00 p.m. Write status report for the day
6:00 p.m. Leave office for home

It shocked me. This person was spending 50% or less of his daily time on testing. This does not even take any ad-hoc meetings, training, test environment down-time or usual interruptions into account. If what this person said is true and it is a good representation of the daily schedules kept by testers in companies, I don't think it is okay. Why?

1. Testing software is the core function of a software test engineer. Most of the time should be spent on this important task.
2. Non-testing tasks e.g. team meetings, email exchanges and compiling test statuses are important but not at the cost of testing time.
3. Time should not be spent on redundant activities. A 10-minute stand-up summarizing what everyone did yesterday is fine but listening to the details of what each of the 10 team members did yesterday is not the best use of time.
4. There is duplicity of effort. If data is accurately entered in the test management/ project management system, it should be sufficient to produce any desired test status report.

However, a test engineer may not be able to guard his time because many of these non-testing tasks are driven by their managers. If a manager insists on long meetings and detailed status reports everyday, which task do you think would suffer? Yes, it is testing that would be impacted. It may not be as deep, wide or inspired as it could have been.

If you are a test manager, you may want to think about how to minimize the burden of non-testing tasks of your testers. Keep them accountable to find problems with the software while allowing them to focus on their core function. Most of the time.

Saturday, February 26, 2011

How to be more successful in software testing interviews?

Hope that you liked my previous post, How to write a resume that is selected? If you are really interested in a position and they have short-listed your resume, the next step is to get selected in the interview :). Here is how to increase your chances of success in interviews.
A couple of thumb rules first:
1. For each interview you attend,
  • Make the effort to understand (I mean internalize, not just read) the job description.
  • Gather more information about the job, the project and the organization.
  • Anticipate questions and prepare excellent answers ahead of the interview.
2. Interview more :)
You should reach the interview venue well in time, go over your notes (see the 1st thumb rule) and relax before the interview starts.


Now is the time to use your preparation, thinking skills and communication skills. Every question you are asked is an opportunity for you to demonstrate your suitability for the position. When you answer a question, do not blurt out the first thing that comes to your mind. Understand the question and frame a very good reply in your mind before speaking. But don't take too long to respond; a pause of few seconds is okay. Provide specific details from your own experience. Aim to surprise the interviewer positively. Here are some examples:

Question. What tests can you design for a common object, say an office chair?
 An excellent response would highlight the test design approach as well as really good test ideas. For example, one could say that I would analyze the documented requirements, designs, contemporary objects, standards and use my own ingenuity to design the tests. Some test ideas would be:
1. Does the chair accommodate the weight of a large person for extended hours and continuous movement? Does the chair allow 360 degree movement, height/ back/ neck/ armrest adjustments? Is the chair comfortable? Ergonomic?
2. Is the chair good to see and feel? Does the color match the office decor? Does it require maintenance/ cleaning infrequently?
3. On being used daily, how long can it stay in perfect condition? What are the steps to install and adjust the chair? and so on.
Question. Can you describe a bug that you discovered?
 A good response would indicate the process used to find the bug and some details of the bug itself. The chosen bug should be a high severity bug (e.g. one impacting financial calculations, one affecting system availability to a large number of users), not caught by the pre-existing test suite and potentially disastrous if released to the customers.

This is the time when you have to judge the success your interview.
If you do not receive outright acceptance, it is possible that there is an objection. You need tact to get this from your interviewer. Possible ways to know the objection are:
1. Ask for the next steps and timelines - If there is an objection, you may get a vague response e.g. "We will get back to you".
2. Summarize each part of the job description and highlight how suitable you are with respect to each part. Note the reaction of the interviewer to each part keenly.
3. If required, be direct and ask for the objection.
 It may not be possible to remove the objection due to say salary/ position fitment, regulatory requirement (e.g. security clearance), customer requirement or specific instructions from top management. However if you truly feel that the objection can be removed, talk about it professionally. Hollow promises will not move a professional interviewer. Give examples from your experience when you faced a similar challenge and how you solved it. State your plan how you would remove the objection quickly.
If there is no objection and you are accepted, good luck with the new offer and congratulations.