Code Coverage Techniques and Tools

Quality of Software can be assured through the Software Testing process of the Software Development Life Cycle (SDLC). With everything from rockets to doorbells running on codes, it is very crucial to ensure that quality code is being delivered with every release. Code Coverage is one of the essential metrics companies use to produce healthier code faster with less risk involved. Techniques involved in Code Coverage have proved to improve testing effectiveness significantly.

What is Code Coverage?

Source

Code Coverage, as the name suggests, is all about measuring how well your tests cover your code. In other words, it describes the degree to which the coding of an application has been tested when a particular test suite runs. It is considered one of the forms of White Box Testing and is usually performed by Developers during Unit Testing. Code coverage scripts generate a report that lists how much of the application code has been executed.

How is Code Coverage measured?

It can be calculated using the formula:

Code Coverage Percentage = (Number of lines of code executed)/(Total Number of lines of code in an application) * 100.

If the test suite executes the entire piece of code, including all the loops, branches, functions, function calls, or conditions, then it can be said that the Code Coverage for that piece of code is 100% which means that a quality product will be delivered to the customer.

Why is Code Coverage important?

Code Coverage is essential because it:

• Helps in maintaining the code base while also letting any features be added later with little-to-no-efforts.
• Helps in exposing bad, dead, and unused codes. Thereby maintaining the product quality.
• Helps in finishing the software development process faster, thus, helping in increasing productivity and efficiency.
• Helps companies to launch more software applications in the market in lesser time.
• Helps to increase customer satisfaction resulting in higher ROI.

Code Coverage Techniques

Following are the major techniques used by companies to measure the lines of code:

1. Statement Coverage

Statement Coverage or Block Coverage measures if all the possible executable statements of code have been executed at least once. This ensures coverage of all possible lines, paths, and statements in the source code. Different input values may have to be used to cover all conditions in the source code since it may have a wide variety of elements, such as operators, looping, functions, exception handlers, etc.

Statement Coverage can be calculated by:

Statement Coverage Percentage = (Number of statements executed)/(Total Number of statements)*100

Consider the below source code example (it contains 7 statements) to calculate Statement Coverage:

Sum (int x, int y) { 

    int result = x + y; 

    If (result> 0)

     Print ("Positive Result", result)

    Else

     Print ("Negative Result", result)

    }

Scenario 1:

x=2, y=3

In the above scenario, the ‘Else’ part of the source code would not get executed since (2 + 3)= positive number. 

Sum (int x, int y) { 

    int result = x + y; 

    If (result> 0)

     Print ("Positive Result", result)

    Else

     Print ("Negative Result", result)

    }

Statement Coverage is equal to (5/7)*100 = 71%

Scenario 2:

x= -2, y= -3

In the above scenario, the ‘If’ part of the code would not get executed since (-2 + (-3))= negative number. 

Sum (int x, int y) { 

    int result = x + y; 

    If (result> 0)

     Print ("Positive Result", result)

    Else

     Print ("Negative Result", result)

    }

Statement Coverage is equal to (6/7)*100 = 85%

This means that with either set of values, our Statement Coverage would not be 100%. In such cases, we may have to execute the tests with all two [(2, 3), (-2, -3)] sets of values to ensure 100% Statement Coverage.

Hence, Statement Coverage covers

• Missing statements
• Unused branches
• Unused statements
• Dead code

2. Decision Coverage

Decision Coverage or Branch Coverage ensures that each and every branch appearing in each of the conditional structures gets executed in the source code at least once. It helps in measuring fractions of independent code segments and finding out sections having no branches. Since Branch Coverage measures execution paths, it has more value over Statement Coverage. Hence, 100% Branch coverage implies 100% Statement coverage. The outcome will be binary in this coverage. Hence, both True and False outcomes must be tested.

Decision Coverage can be calculated by:

Decision Coverage Percentage = (Number of decision/branch outcomes executed)/(Total number of decision outcomes in the source code)*100

Consider the below source code example to calculate Decision Coverage:

Sample (int x) { 

    If (x >= 5)

x = x * 2

    Print (x)

    }

Scenario 1:

x= 2

In the above scenario, the outcome = No since 2 <= 5.

Sample (int x) { 

    If (x >= 5)

x = x * 2

    Print (x)

    }

Decision  Coverage = 50%

Scenario 2:

x= 7

In the above scenario, the outcome = Yes since 7 >= 5.

Sample (int x) { 

    If (x >= 5)

x = x * 2

    Print (x)

    }

Decision  Coverage = 50%

This means that with either set of values, our Decision Coverage would not be 100%. In such cases, we may have to execute the tests with both the values – 2 and 7 to ensure 100% Decision Coverage.

Hence, Decision Coverage covers

• All branches – If, Else, While, Do
• Removes issues happening due to Statement Coverage testing

3. Function Coverage

Function Coverage ensures that all the necessary functions present in the source code are covered during test execution. These functions need to be tested for varying values so that they get tested thoroughly. In the source code, there may be multiple functions, and depending on the input values used, they may or may not be called. Thus, the purpose of Function Coverage is to ensure that we have each function called for.

Function Coverage can be calculated by:

Function Coverage Percentage = (Number of functions called)/(Total number of functions)*100

Consider the below source code example to calculate Function Coverage:

Add (int x, int y) { 

    int result = x + y; 

    If (result> 0)

     Print ("Positive", result)

    Else

     Print ("Negative", result)

    }

4. Condition Coverage

Condition Coverage or Expression Coverage is used to test and evaluate the variables or sub-expressions in the conditional statement. It ensures that the tests cover both the conditional statement values, i.e., true or false. It also helps to provide proper coverage to the control flow. It offers better sensitivity to the control flow than decision coverage. In this coverage, expressions with logical operands are only considered.

Condition Coverage can be calculated by:

When each occurring condition is evaluated for both true and false states in the source code, the Condition Coverage would be 100%. If an expression has Boolean operations like AND, OR, or XOR, it indicates total possibilities.

Consider the below example to calculate Condition Coverage:

If (x < y) AND (c > d) THEN

For the above expression, there are 4 possible combinations: TT, TF, FT, FF

Scenario:

x=3, y=4

c=3, d=4

x<y = TRUE, c<d = FALSE which satisfies one of the 4 combinations (TF).

Therefore, Condition Coverage is equal to (1/4)*100 = 25%

Condition Coverage covers

• All the conditional statements in the source code
• All the logical expressions in the source code

If our tests call the ‘Add’ function even once, then we would call this as a 100% Function Coverage.

Hence, Function Coverage covers

• All the functions in the source code
• All the subroutines in the source code

Which type of Code Coverage to choose?

This depends mostly on the code under testing. The higher the probability of defects causing costly production failures, the more severe the level of code coverage should be chosen. The developer/tester should also check the cost of potential penalties, multiple undiscovered defects, lost sales, etc.

Advantages of using Code Coverage

• It helps in finding new uncovered test cases.
• It helps in creating extra test cases to increase coverage.
• It helps in measuring the test implementation efficiency.
• It helps in the exposure of bad, dead, and unused code.
• It helps in determining the quality and performance aspects of any software.
• It helps in developing the software product faster by increasing its productivity and efficiency.

Disadvantages of using Code Coverage

• It can not guarantee that all the possible test cases or values of a feature are tested.
• Aiming for a 100% code coverage can cause a lack of robustness in the tests resulting in missing out on capturing the defect-prone scenarios.
• It is impossible to determine how perfectly the code has been covered.

Code Coverage Tools

There are several tools available in the market to measure the Code Coverage of any Software. Some are given below:

• Cobertura
• Clover
• Gretel
• Kalistick
• JaCoCo
• JTest
• OpenCover
• Emma
• GCT

Conclusion

In this rapidly growing technology-driven world, testers and developers have to minimize their software development life cycles along with producing high-quality software for the customer. In order to handle such tight deadlines, software engineers must build only good code. Hence, good quality code is what every developer or tester should be aiming for. 

With a code coverage analysis report, they can track the proportion of code that worked well under different testing scenarios. This insight will act like a feedback report, thereby helping developers to write good and clean source code. This will ultimately result in improved code quality, positively impacting the software quality.