1.13. Mutation Testing Examples¶
1.13.1. Types of Mutants¶
The main purpose of writing test cases is to make sure that your solution code is working properly and is producing the intended behavior that you want. There are different mutators in mutation testing that allows you to check if your test suite is properly testing every element in your project solution code. This is a great way to make sure that your solution code is working according to your intended logic.
Another benefit of using mutation testing is to make sure your solution code has good design and abides by good programming practices. This means your solution code does not have unnecessary complexities, dead codes, bad design, technical debts, etc.
There are different mutators that ensure different qualities of a programming project. For the programming projects in this course, we are going to use only two mutators to generate the following mutants:
Arithmetic Operation Mutant
Logical Expression Mutant (Remove Conditionals)
Don’t worry, you will not have to set anything. If you have properly intalled the latest version of the Web-CAT Submission Plug-in, then the mutation testing plug-in in your Eclipse IDE should come with the desired two mutators set as default.
In the following sections, we will look into:
What these two mutators are?
How they work?
What feedback do they produce?
How to use the HINT in the feedback to solve the mutations?
How to write test cases to improve mutation coverage?
Different examples on these two mutators.
1.13.1.1. Arithmetic Operation Mutant¶
The arithmetic operation mutant checks if an arithmetic operation is being tested by your test suite. This mutator replaces an arithmetic operation with one of its members. The mutator is composed of 2 sub-mutators, AOD_1 and AOD_2, that mutate the operation to its first and second member respectively.
If the test suite detects the change i.e. there is a mutant in the code, then the test suite passes. This results in a higher mutation coverage. Otherwise the test suite fails to detect the mutant. This results in a lower mutation coverage. In such case, we need to write additional test case assertions to test for the arithmetic operation in question.
For example
1int a = b + c;
will be mutated to
1int a = b; // second member replaced
and to
1int a = c; // first member replaced
Here, b = first member and c = second member.
This way, mutation testing ensures that the arithmetic operator is tested for its intended values.
1.13.1.2. Example Code 1: Arithmetic Operation Mutant¶
As an example, we want to write a function that takes two numbers and returns the summation of both numbers.
1 public static int Addition(int num1, int num2) {
2 int sum = 0;
3
4 sum = num1 + num2; // --> math operation
5
6 return sum;
7 }
Now, if we execute the mutation testing on our newly written function without writing any test cases, it will mutate the code as follows:
# Replacing the first member in the arithmetic operation:
1 public static int Addition(int num1, int num2) {
2 int sum = 0;
3
4 sum = num2; // --> math operation
5
6 return sum;
7 }
# Replacing the second member in the arithmetic operation:
1 public static int Addition(int num1, int num2) {
2 int sum = 0;
3
4 sum = num1; // --> math operation
5
6 return sum;
7 }
As a result, it will generate the following mutations in the LINES_NOT_TESTED group under the Mutations List tab.
The icons for unresolve mutants are shown as “red bugs”.

Figure 1.13.1: Example Code 1: Arithmetic Operation Mutant without test cases¶
Here, we can see that the HINT suggests us to write test case assertions to test the arithmetic operation for intended behavior.
In order to fix the mutations, we can write the following test case assertions:
1 // testAddition tests for adding two numbers
2 @Test
3 public void testAddition() {
4 // testing if 5+10 == 15
5 assertEquals(15, SimpleExample.Addition(5, 10));
6 }
If we run the mutation testing again then we get the mutations in the LINES_WITH_GOOD_TESTING group under the Mutations List tab.
The icons for resolved mutants are changed to “green bugs”.

Figure 1.13.2: Example Code 1: Arithmetic Operation Mutant with test cases¶
1.13.1.3. Logical Expression Mutant (Remove Conditionals)¶
The logical expression mutator (a.k.a. remove conditionals mutator) checks if a conditional expression is properly tested by your test suite. This mutator replaces the conditional expression with either TRUE or FALSE and then runs your test suite with the mutant.
If the test suite detects the change i.e. there is a mutant in the code, then the test suite passes. This results in a higher mutation coverage. Otherwise the test suite fails to detect the mutant. This results in a lower mutation coverage. In such case, we need to write additional test case assertions to test for the conditional expression in question.
For example replacing conditional expression with TRUE condition:
1if (a == b) {
2// do something
3}
will be mutated to
1if (true) {
2// do something
3}
For example replacing conditional expression with FALSE condition:
1if (a == b) {
2// do something
3}
will be mutated to
1if (false) {
2// do something
3}
If there are more than one conditional expressions then each expression will be mutated in separate runs of the test suite. The conditional mutator also mutates the bytecode instructions for order checks (e.g. <=, >).
If there are more than one conditional expression in the same statement, then the generated mutants will be in order of the conditional expressions in the statement. Keep in mind, for multiple conditional expressions, you must test each and every one of the expressions.
1.13.1.4. Example Code 2: Logical Expression Mutant (Remove Conditionals)¶
As an example, we want to write a function that takes a number and returns TRUE if the number is positive (greater than zero) and FALSE if the number is otherwise.
1 public static boolean PositiveCheck(int number) {
2 if (number > 0) { // --> true or false (2 cases)
3 return true; // positive number
4 }
5 else {
6 return false; // zero or negative number
7 }
8 }
Now, if we execute the mutation testing on our newly written function without writing any test cases, it will mutate the code as follows:
# Replacing the conditional expression with TRUE:
1 public static boolean PositiveCheck(int number) {
2 if (true) { // --> true or false (2 cases)
3 return true; // positive number
4 }
5 else {
6 return false; // zero or negative number
7 }
8 }
# Replacing the conditional expression with FALSE:
1 public static boolean PositiveCheck(int number) {
2 if (false) { // --> true or false (2 cases)
3 return true; // positive number
4 }
5 else {
6 return false; // zero or negative number
7 }
8 }
As a result, it will generate the following mutations in the LINES_NOT_TESTED group under the Mutations List tab.
The icons for unresolve mutants are shown as “red bugs”.

Figure 1.13.3: Example Code 2: Logical Expression Mutant (Remove Conditionals) without test cases¶
In order to fix the mutations, we can write the following test case assertions:
1 // testEvenOddCheckWithEvenNumber tests for positive number
2 @Test
3 public void testPositiveCheckWithPositiveNumber() {
4 assertTrue(SimpleExample.PositiveCheck(10));
5 }
6
7 // testEvenOddCheckWithOddNumber tests for zero
8 @Test
9 public void testPositiveCheckWithZero() {
10 assertFalse(SimpleExample.PositiveCheck(0));
11 }
12
13 // testEvenOddCheckWithOddNumber tests for negative number
14 @Test
15 public void testPositiveCheckWithNegativeNumber() {
16 assertFalse(SimpleExample.PositiveCheck(-5));
17 }
If we run the mutation testing again then we get the mutations in the LINES_WITH_GOOD_TESTING group under the Mutations List tab.
The icons for resolved mutants are changed to “green bugs”.

Figure 1.13.4: Example Code 2: Logical Expression Mutant (Remove Conditionals) with test cases¶
1.13.1.5. Example Code 3: Multiple Mutants in One (EvenOddCheck)¶
We can have programming statements where we have both arithmetic operation(s) and conditional expression(s). In such cases, mutation testing will return mutants for each type and list them under the Mutations List tab.
For example, we want to write a function that takes a number and returns TRUE if the number is even and FALSE if the number is odd.
1 public static boolean EvenOddCheck(int number) {
2 if (number % 2 == 0) { // --> true or false (2 cases)
3 return true; // even number
4 }
5 else {
6 return false; // odd number
7 }
8 }
As a result, it will generate the following mutations under the Mutations List tab:

Figure 1.13.5: Example Code 3: Multiple Mutants in One Statement without test cases¶
In order to fix the mutations, we can write the following test case assertions:
1 // testEvenOddCheckWithEvenNumber tests for even number
2 @Test
3 public void testEvenOddCheckWithEvenNumber() {
4 assertTrue(SimpleExample.EvenOddCheck(10));
5 }
6
7 // testEvenOddCheckWithOddNumber tests for odd number
8 @Test
9 public void testEvenOddCheckWithOddNumber() {
10 assertFalse(SimpleExample.EvenOddCheck(5));
11 }
If we run the mutation testing again then we get the following mutations under the Mutations List tab.

Figure 1.13.6: Example Code 3: Multiple Mutants in One Statement with test cases¶
1.13.1.6. Example Code 4: Loop Conditions (optional)¶
A loop contains a conditional expression that needs testing. However, this is often optional and does not negatively impact mutation coverage.
For example:
1for (int i = 0; i < 10; i++)
If you don’t test the terminating condition of a for loop then you can get the mutant with the following feedback in the TIMED_OUT group under the Mutations List tab.

Figure 1.13.7: Example Code 4: Loop Conditions¶