Orthogonal Array Testing in Software Engineering | Orthogonal Array Testing Example

Welcome to this post on Orthogonal Array Testing in software engineering. Orthogonal array testing is black box testing. When you have a large number of input combinations, orthogonal array testing gives very few test cases.

What is Orthogonal Array? It is a table. The columns (a.k.a. factors) represent the independent variables. The rows (a.k.a. experiments or runs) represent the variables' combinations. The orthogonal array testing example below is for a web page with three sections - TOP, LEFT and RIGHT. Note that the orthogonal array has only 4 test cases to run for 8 input combinations.

What is Orthogonal Array Testing? The Orthogonal Array Technique has the following steps:

1. Identify the independent variables. Put them as column headers in the table.
   
2. For each independent variable, identify the number of possible values.
3. Search an Orthogonal Array with the smallest number of rows in orthogonal array design of experiments. I have explained this step here.
   
4. Put the independent variable values in the Orthogonal Array cells. If any cells are still blank, repeat the values in them.
5. Test each row in the table.
   

In the example below, there are 3 independent variables (put as column headers). Variable A has 2 possible values, A1 and A2. Variable B has 3 possible values, B1, B2 and B3. Variable C has 3 possible values, C1, C2 and C3. The input combinations are therefore 2 * 3 * 3, which is 18. The smallest number of rows can be found in this orthogonal array. Using the above orthogonal array technique, the table should look like below. Instead of testing 18 input combinations, orthogonal array gives only 9 test cases to run.

Note: 1) Since variable A has only 2 values, they are used to complete the first column.
2) Since there are only 3 variables, the 4th column is not used.

Want to learn the above Orthogonal Array Testing examples in detail? Please view my Orthogonal Array Testing tutorial. Thank you.

Mutation Testing in Software Testing | Mutation Analysis | White Box Testing

After being occupied with some commitments for about two months, I finally got time to write the next article 😊. This post is on Mutation Testing, a white box testing, to test program code. Mutation testing is also useful for test automation code, databases, software models and other artifacts in software engineering

What is Mutation Testing in software testing? Basically, in Mutation Testing, you make a change (a mutation) to your program and run your tests with test data. Mutation Testing finds if your existing tests and test data are useful or not. Using Mutation Testing, you could know which sections of your program are tested poorly. Also, you could identify your tests that never find any mutants (changed copies of your program). In the example below, the program on the left is the original program and the program on the right is it's mutant.

Mutation Testing process (Mutation Analysis)

1) You test your original program with all your tests and test data. If any test fails, you need to fix your program or that test or it's test data. 

2) Once your program passes all the tests, you create mutants by using any mutation operator (e.g delete a statement, duplicate a statement, exchange operators etc.) and test your mutants. Each test run on any mutant should ideally fail, because a mutant is a changed copy of your program. 

3) If a test run on any mutant passes, you should find out the reason (e.g. the mutated code is not run or the mutant is functionally "equivalent" to your program). In order to find more mutants, you need to update your existing tests (or test data) and/ or write new tests (with test data). Also, if you update your test set or test data, you need to repeat the above process from step 1).

Want to learn Mutation Testing more? Like Mutation Analysis in detail, Mutation Score and Mutation Testing assumptions? Then, please view the complete Mutation Testing tutorial. Thank you.

Data Flow Testing in Software Testing | White Box Testing

This post is on Data Flow Testing. What is data flow testing? Data flow testing in software testing is white box testing. Data flow testing is also a structured testing methodology, based on your program's internal structure. Data flow testing focuses on variables' definition and use. A variable is a memory location that can store a value. A variable's value can be used in a condition or a calculation. Now, in a program, it is possible for the developer to use a variable that doesn't exist or update a variable's value by error. In data flow testing, you examine the variables in your program. View my Data Flow Testing tutorial for more details or read on.

Data flow testing strategies in software testing include:
1) Define Use Testing - DU Testing has some rules and metrics. It tests the paths of the Control Flow Graph (view my Path Testing tutorial to learn about Control Flow Graph) that have a variable defined or used.
2) Program Slices - Program slices strategy divides the program into executable sections, for a variable. You could then test a single program slice independently, for that variable.

Let us see the data flow testing example. This example program computes the Sum of some numbers. Here is the Control Flow Graph when the variable n gets a value of 0 (in Node 4).

If the variable n has a non-zero value, a different path is taken in the Control Flow Graph. The reason is that the while loop is also run. Here is the Control Flow Graph when the variable n gets a value of say, 100, the first time (in Node 4) and 0 the second time (in Node 8).

Let us now learn about Define Use Testing. A particular node in the Control Flow Graph can be either a Define Node or Use Node. A variable is given a value in a Define Node. Examples of Define Nodes are Node 2 (the variable i is given a value), Node 3 (the variable sum is given a value), Node 4 (the variable n is given a value) or Node 8 (again, the variable n is given a value). A variable's value is used in a Use Node. Examples of Use Nodes are Node 6 (the value of variable sum is used; also the value of variable n is used) or Node 7 (the value of variable i is used in the computation) or Node 10 (the value of variable i is used; also the value of variable sum is used in the print statement). Learn about P-use (Predicate Use) and C-use (Computation Use) here. In Define Use Testing, you should identify the Define Use nodes for each variable. I have explained how to find the Define nodes and Use nodes for variables i, n and sum here. In Define Use Testing, you should identify some paths according to your chosen Define Use Testing Metrics and test those paths to find bugs.

 

Moving on to Program Slices. This data flow testing strategy is useful when your program is large in size. You could identify a program slice (a set of statements) for a single variable, up to any statement in your program. I have explained how to identify program slices for variables i, n and sum here.

Using both Define Use Testing and Program Slices data flow testing techniques, here are four Define Use Paths (pink table below) to test for variable n. In Data Flow Testing, you could test these paths. Similarly, you could identify the paths to test for variable i and variable sum and test them too.

Want to learn Data Flow Testing in detail? Learn Data Flow Testing Metrics? Then please view the complete Data Flow Testing with Example Tutorial. Thank you.