App & Browser Testing Made Easy

Give your users a seamless experience by testing on 3000+ real devices and browsers. Don't compromise with emulators and simulators

Get Started free
Home Guide What is Fault Injection in Software Testing?

What is Fault Injection in Software Testing?

By Vasanth DS, Community Contributor -

Software Fault Injection deliberately injects faults into a software system to determine whether the system can withstand error conditions. It is a practice of stress testing or monkey testing the software by injecting faults that result in disruptive events, observing how the software responds to the events and implementing improvements.

Fault Injection is a complementary technique to software testing for improving software performance and resiliency. The usual testing methods validate the correct behavior of the software, whereas Fault Injection evaluates the correct behavior of software with faults that are injected intentionally. Let’s understand further why fault Injection is a complementary technique in software testing for improving software performance and resiliency.

Software Fault Injection Mechanism

In the case of software testing, the faults are injected into software to accomplish the following:

  • Evaluate if the software can withstand and recover from error conditions
  • Identify potential defects or security vulnerabilities before production 
  • Evaluate software robustness.
  • Improve the design of the system for resilience and performance

The Software Fault Injection works based on the Fault-Error-Failure-Cycle mechanism. The cycle consists of the following steps.

  1. Fault: Faults (adjusted or hypothesized) are injected into the code either in compile time or run time.
  2. Error: The faults, when activated, lead to multiple errors and change the software’s state to an undesired state.
  3. Failure: The undesired state caused due to multiple errors that fail the software. For example, outage of service.

Fault Injection Mechanism

Source

Software Fault Injection Types

Fault Injection can be categorized into two types:

  • Compile-time Fault Injection: A fault is injected into the source code during the compile time. The code changes can include the addition or modification of a code.
  •  Run-time Fault Injection—A fault is injected into software while running. A fault can be injected dynamically to external libraries or added to other dependencies during runtime. The run-time Fault Injection can be further classified as:
    • Time-based trigger— A trigger is set to inject a fault at a specified time.
    • Interrupt-based trigger— A trigger is set to inject using a specific location in the code or event in the system. 

The following table lists the differences between Compile-time and Run-time Fault Injections:

Compile-time InjectionRuntime Injection
Faults are introduced before the run-time.Faults are introduced during the runtime.
Source code is modified to inject errors.Additional code is added to inject errors.
Code mutation and code insertion.Time-based or interrupt-based triggers.

Fault Injection Environment

The Fault Injection environment typically consists of the following components.

  • Workload Generator: Contains software workloads.
  • Fault Injector: Injects faults into the target software while the commands are executed from the workload generator.
  • Monitor: Monitors the execution of the commands and collects data as required.
  • Data Collector: Performs online data collection.
  • Data Analyzer: Performs data processing and analysis.
  • Controller: Controls the experiment. The controller is a program that can run on the target software system or on a separate computer. 

Fault Injection Environment

Source

Compile-time Fault Injection Examples

This section lists some examples of how faults are injected during compile time by modifying the code. The code injected through this method results in errors similar to the errors that the programmers unintentionally commit.

Example: Code Modification

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a - 1;
}
return 0;
}

Modified Code:

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a + 1; // '-' is changed to '+'
}
return 0;
}

In the modified code, a fault is injected by modifying the code from “a=a-1” to “a=a+1”. The variable “a” value increases and never meets the while condition. Therefore, the while loop goes into an infinite loop and never ends.

Example: Code Insertion

The following example shows how a fault is injected during the compile time by inserting the code instead of modifying the code. In this case, an additional code is added to change the parameter’s value or value.

Original Code:

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a - 1;
}
return 0;
}

Modified Code:

int main()
{
int a = 10;
while ( a > 0 )
{
cout << "GFG";
a = a - 1;
a++; // Additional code
}
return 0;
}

In this example, an additional line of code “a++” is added to change the value of the variable “a”. 

Run-time Fault Injection Example

The following figure shows the exception triggered when a fault is injected on a dummy .NET WinForm application named TwoCardPokerGame.exe. A C# program that runs in the backend at run time alters the behavior of the software when you click the Evaluate button. In this situation, the Two Card Poker application cannot handle the exception and displays an error message.

 

Run time Fault Injection Example

Source

Sample Fault Injection code:

using System;
namespace FaultHarness {
class Program {
static void Main(string[] args) {
try {
Console.WriteLine("\nBegin TestApi Fault Injection environmnent session\n");
// create fault session, launch application
Console.WriteLine("\nEnd TestApi Fault Injection environment session");
}
catch (Exception ex) {
Console.WriteLine("Fatal: " + ex.Message);
}
}
} // class Program
} // ns

When to use Fault Injection in Software Testing?

Fault Injection is a recommended approach to assess the fault tolerance of software with multiple dependent components. The software can be considered fault tolerant only when all dependent software layers and components are fault tolerant.

Fault Injection testing can be used when your software is built using one or more third-party software or APIs and if it is deployed in a multi-platform or cross-platform environment.  The software with such dependencies is prone to dependency disruptions. If an API used in the software fails, it propagates the failure to the dependent component in the software.  Fault Injection is an acknowledged method for assessing the dependability of software systems. It can also evaluate software during the initial stage of the SDLC.

Fault Injection tools

Numerous tools are available in the market, using which you can inject a fault into your software. The following are the widely used tools.

  •  Xception
  •  beStorm
  •  Holodeck
  •  Grid-FIT
  •  Orchestra
  •  ExhaustiF
  •  The Mu Service Analyzer

Follow-Up Read: Essential Tools for Remote Software Testing

Advantages and Disadvantages of Software-based Fault Injection

Advantages

Disadvantages

Does not require any specialized hardware

Can run near real-time scenarios

Simple and low cost for implementation

Can add a new class of faults later

Enables engineers to see the aftereffects of bugs or mistakes before they happen normally in production.

Enables engineers to make changes to previously unknown issues before discharge.

Cannot inject faults into locations not accessible to software

Controllability is very limited

Requires modification of original source code to inject faults

Near impossible to model permanent faults.

The software instrumentation may disturb the workload running on the target system and event.

Changing the structure of the original software

Summary

Software Fault Injection is a recommended approach to evaluate the tolerance and reliability of the software. Primarily, it is a promising approach to evaluate the software that is deployed in a complex and cross-platform environment. However, with fault injection, it’s crucial to test on a real device cloud that facilitates manual and automated testing on the most-preferred testing infrastructure. 

On the BrowserStack infrastructure, you can access: 

Sign Up & Start Testing

Tags
Automation Testing Real Device Cloud

Featured Articles

Bug Severity vs Priority in Testing

Data Visualization for Better Debugging in Test Automation

App & Browser Testing Made Easy

Seamlessly test across 20,000+ real devices with BrowserStack