Katalon The State of Quality Report 2022 Review and Download

Katalon has made The State of Quality Report 2022. This helpful report has important findings about Quality, Test Automation, Artificial Intelligence (AI) in Test Automation and the year 2022 for Quality Assurance. You can view a summary of this valuable report here or read on.

Katalon provides test automation tools and platform that can be integrated with other products. Katalon is well-known in the industry. Katalon tools are used in more than 100K enterprises by more than 1 Million testers and developers.

I explained the summary of The State of Quality Report 2022 in my 6-minute video above. But in brief, The State of Quality Report 2022 has four parts. These are Quality at speed, Test Automation, AI in Test Automation and 2022 expectations. In Quality at speed section, Katalon suggests test automation, code review, production testing, automated unit testing and shift left testing. In Test Automation section, Katalon found that test automation is used for regression testing, test cases generation, functional testing, test results analysis, test data generation, performance testing etc. Katalon also names the most popular test automation tools. In AI in Test Automation section, Katalon found that AI is in initial stages only. AI is being used for test script generation, test data generation, automated defect detection and test selection. AI in test automation is expected to increase. In 2022, Katalon expects that test automation would increase, QA practices would improve and AI technologies would enable tasks like test estimation, test repair etc.

Benefits to you? This 59-page report explains the quality state in detail. There are many recommendations that you may discuss with your QA team or suggest in interviews. You can download The State of Quality Report 2022 for free at the link, https://bit.ly/3fPXLgb

* You also get a chance to win a $30 E-gift card.

Orthogonal Array Testing in Software Engineering | Orthogonal Array Testing Example

Welcome to this post on Orthogonal Array Testing in software engineering. Orthogonal array testing is black box testing. When you have a large number of input combinations, orthogonal array testing gives very few test cases.

What is Orthogonal Array? It is a table. The columns (a.k.a. factors) represent the independent variables. The rows (a.k.a. experiments or runs) represent the variables' combinations. The orthogonal array testing example below is for a web page with three sections - TOP, LEFT and RIGHT. Note that the orthogonal array has only 4 test cases to run for 8 input combinations.

What is Orthogonal Array Testing? The Orthogonal Array Technique has the following steps:

1. Identify the independent variables. Put them as column headers in the table.
   
2. For each independent variable, identify the number of possible values.
3. Search an Orthogonal Array with the smallest number of rows in orthogonal array design of experiments. I have explained this step here.
   
4. Put the independent variable values in the Orthogonal Array cells. If any cells are still blank, repeat the values in them.
5. Test each row in the table.
   

In the example below, there are 3 independent variables (put as column headers). Variable A has 2 possible values, A1 and A2. Variable B has 3 possible values, B1, B2 and B3. Variable C has 3 possible values, C1, C2 and C3. The input combinations are therefore 2 * 3 * 3, which is 18. The smallest number of rows can be found in this orthogonal array. Using the above orthogonal array technique, the table should look like below. Instead of testing 18 input combinations, orthogonal array gives only 9 test cases to run.

Note: 1) Since variable A has only 2 values, they are used to complete the first column.
2) Since there are only 3 variables, the 4th column is not used.

Want to learn the above Orthogonal Array Testing examples in detail? Please view my Orthogonal Array Testing tutorial. Thank you.

Mutation Testing in Software Testing | Mutation Analysis | White Box Testing

After being occupied with some commitments for about two months, I finally got time to write the next article 😊. This post is on Mutation Testing, a white box testing, to test program code. Mutation testing is also useful for test automation code, databases, software models and other artifacts in software engineering

What is Mutation Testing in software testing? Basically, in Mutation Testing, you make a change (a mutation) to your program and run your tests with test data. Mutation Testing finds if your existing tests and test data are useful or not. Using Mutation Testing, you could know which sections of your program are tested poorly. Also, you could identify your tests that never find any mutants (changed copies of your program). In the example below, the program on the left is the original program and the program on the right is it's mutant.

Mutation Testing process (Mutation Analysis)

1) You test your original program with all your tests and test data. If any test fails, you need to fix your program or that test or it's test data. 

2) Once your program passes all the tests, you create mutants by using any mutation operator (e.g delete a statement, duplicate a statement, exchange operators etc.) and test your mutants. Each test run on any mutant should ideally fail, because a mutant is a changed copy of your program. 

3) If a test run on any mutant passes, you should find out the reason (e.g. the mutated code is not run or the mutant is functionally "equivalent" to your program). In order to find more mutants, you need to update your existing tests (or test data) and/ or write new tests (with test data). Also, if you update your test set or test data, you need to repeat the above process from step 1).

Want to learn Mutation Testing more? Like Mutation Analysis in detail, Mutation Score and Mutation Testing assumptions? Then, please view the complete Mutation Testing tutorial. Thank you.