Mutation Testing in Software Testing | Mutation Analysis | White Box Testing

After being occupied with some commitments for about two months, I finally got time to write the next article 😊. This post is on Mutation Testing, a white box testing, to test program code. Mutation testing is also useful for test automation code, databases, software models and other artifacts in software engineering. View the below tutorial or read on...

What is Mutation Testing in software testing? Basically, in Mutation Testing, you make a change (a mutation) to your program and run your tests with test data. Mutation Testing finds if your existing tests and test data are useful or not. Using Mutation Testing, you could know which sections of your program are tested poorly. Also, you could identify your tests that never find any mutants (changed copies of your program). In the example below, the program on the left is the original program and the program on the right is it's mutant.

Mutation Testing process (Mutation Analysis)

1) You test your original program with all your tests and test data. If any test fails, you need to fix your program or that test or it's test data. 

2) Once your program passes all the tests, you create mutants by using any mutation operator (e.g delete a statement, duplicate a statement, exchange operators etc.) and test your mutants. Each test run on any mutant should ideally fail, because a mutant is a changed copy of your program. 

3) If a test run on any mutant passes, you should find out the reason (e.g. the mutated code is not run or the mutant is functionally "equivalent" to your program). In order to find more mutants, you need to update your existing tests (or test data) and/ or write new tests (with test data). Also, if you update your test set or test data, you need to repeat the above process from step 1).

Want to learn Mutation Testing more? Like Mutation Analysis in detail, Mutation Score and Mutation Testing assumptions? Then, please view the complete Mutation Testing tutorial. Thank you.

Data Flow Testing in Software Testing | White Box Testing

This post is on Data Flow Testing. What is data flow testing? Data flow testing in software testing is white box testing. Data flow testing is also a structured testing methodology, based on your program's internal structure. Data flow testing focuses on variables' definition and use. A variable is a memory location that can store a value. A variable's value can be used in a condition or a calculation. Now, in a program, it is possible for the developer to use a variable that doesn't exist or update a variable's value by error. In data flow testing, you examine the variables in your program. View my Data Flow Testing tutorial for more details or read on.

Data flow testing strategies in software testing include:

1) Define Use Testing - DU Testing has some rules and metrics. It tests the paths of the Control Flow Graph (view my Path Testing tutorial to learn about Control Flow Graph) that have a variable defined or used.

2) Program Slices - Program slices strategy divides the program into executable sections, for a variable. You could then test a single program slice independently, for that variable.

Let us see the data flow testing example. This example program computes the Sum of some numbers. Here is the Control Flow Graph when the variable n gets a value of 0 (in Node 4).

If the variable n has a non-zero value, a different path is taken in the Control Flow Graph. The reason is that the while loop is also run. Here is the Control Flow Graph when the variable n gets a value of say, 100, the first time (in Node 4) and 0 the second time (in Node 8).

Let us now learn about Define Use Testing. A particular node in the Control Flow Graph can be either a Define Node or Use Node. A variable is given a value in a Define Node. Examples of Define Nodes are Node 2 (the variable i is given a value), Node 3 (the variable sum is given a value), Node 4 (the variable n is given a value) or Node 8 (again, the variable n is given a value). A variable's value is used in a Use Node. Examples of Use Nodes are Node 6 (the value of variable sum is used; also the value of variable n is used) or Node 7 (the value of variable i is used in the computation) or Node 10 (the value of variable i is used; also the value of variable sum is used in the print statement). Learn about P-use (Predicate Use) and C-use (Computation Use) here. In Define Use Testing, you should identify the Define Use nodes for each variable. I have explained how to find the Define nodes and Use nodes for variables i, n and sum here. In Define Use Testing, you should identify some paths according to your chosen Define Use Testing Metrics and test those paths to find bugs.

 

Moving on to Program Slices. This data flow testing strategy is useful when your program is large in size. You could identify a program slice (a set of statements) for a single variable, up to any statement in your program. I have explained how to identify program slices for variables i, n and sum here.

Using both Define Use Testing and Program Slices data flow testing techniques, here are four Define Use Paths (pink table below) to test for variable n. In Data Flow Testing, you could test these paths. Similarly, you could identify the paths to test for variable i and variable sum and test them too.

Want to learn Data Flow Testing in detail? Learn Data Flow Testing Metrics? Then please view the complete Data Flow Testing with Example Tutorial. Thank you.

Path Testing in Software Testing with Example | Triangle Problem

This post explains Path Testing in software testing. Path testing is one of the test design techniques. Path testing is a structured testing methodology, meaning it is based on mathematics. Path testing is white box testing and complementary to other software testing methodologies like requirements based testing. View my Path Testing tutorial to understand path testing. Then read on.

 

In order to use path testing techniques, you should know about the Control Flow Graph, which represents the program or the algorithm to be tested. Here is the Control Flow Graph for the Triangle Problem example below. In this Control Flow Graph, 1 is the Entry Node and 10 is the Exit node. The nodes 1 through 10 represent the Steps below. The directional arrows e.g. Node 1 to Node 2 represent the control flow between the Steps. The directional arrows are called Edges. Using the Control Flow Graph, you can compute the Cyclomatic Complexity, which is E - N + 2, where E is the number of Edges and N is the number of Nodes. In this Control Flow Graph example, Cyclomatic Complexity is 12 - 10 + 2 = 4.

 

The Triangle Problem is to find out the type of the triangle, given its' three sides. Here is the algorithm for the Triangle Problem.

 

In path testing for triangle problem, there are four basis (independent) paths of algorithm execution, which you need to test with test data.

• Not a Triangle: Node1 > Node2 > Node3 > Node10
• Equilateral Triangle: Node1 > Node2 > Node4 > Node5 > Node10
• Isosceles Triangle: Node1 > Node2 > Node4 > Node6 > Node7 > Node10
• Scalene Triangle: Node1 > Node2 > Node4 > Node6 > Node8 > Node9 > Node10
  

Want to learn Path testing with example explained in detail? Want to learn the Path Testing process? Then, please view my Path Testing in Software Testing tutorial. Thank you.