End To End Testing: A Detailed Guide
Shreya Bose, Technical Content Writer at BrowserStack - February 5, 2020
What is End to End Testing?
End to end testing (E2E testing) refers to a software testing method that involves testing an application’s workflow from beginning to end. This method basically aims to replicate real user scenarios so that the system can be validated for integration and data integrity.
Essentially, the test goes through every operation the application can perform to test how the application communicates with hardware, network connectivity, external dependencies, databases, and other applications. Usually, E2E testing is executed after functional and system testing is complete.
Steps for End to End Testing
The steps below are required to initiate and complete any E2E test.
- Analyze requirements. Have a clear idea of how the app is supposed to work in every aspect
- Set up a test environment in alignment with all the requirements
- Analyze software and hardware requirements
- List down how every system needs to respond
- List down testing methods required to test these responses. Include clear descriptions of standards (language, tools, etc.) to be followed in each test.
- Design test cases
- Run tests, study and save results
End to End Testing Example
Let’s say testers have to verify the functioning of a Gmail account. The following features have to be tested:
- Type URL into the address bar to launch the Gmail login page.
- Log into account with valid credentials.
- Access Inbox. Open Read and Unread emails.
- Compose a new email.
- Reply to and forward an existing email.
- Open Sent items folder. Check emails there.
- Open Spam folder. Check emails there.
- Log out of Gmail by clicking ‘logout’.
E2E Testing must include the following three categories of activity:
1. User Functions
To build user functions, do the following:
- List the features of the software and its interconnected sub-systems.
- For each function, track and record all actions performed. Do the same for all input and output data.
- Identify all relations between user functions.
- Establish if each user function is independent or reusable.
To build conditions based on user functions, do the following:
- Decide a set of conditions for every user function. This could include timing, data conditions, etc. – essentially any factor that can affect user functions.
3. Test Cases
To build test cases for E2E Testing, keep the following in mind:
- Create multiple test cases to test every functionality of user functions.
- Assign at least a single, separate test case to every condition.
Relevant factors to measure End to End Testing
Consider the following factors to track the progress of end to end tests
- Test case preparation: Are the right test cases being created to test every feature of a software? The graph representing this should show the rate at which current vs planned test cases are being crafted.
- Weekly test: A percentage base report of test completion each week is extremely necessary. The important parameters here are currently executed, not executed, and failed test cases.
- Defect Details: A percentage of defects that are identified, closed, or opened each week is important. By distributing defects (for resolution) based on priority, it helps testers and managers keep track of bugs and supervise their resolution.
- Test environment availability: This provides a comparison of the amount of time that is scheduled for testing every day vs the time actually spent.
Main Challenges in End to End Testing
Naturally, detecting bugs in a complex workflow entails challenges. The two major ones are explained below:
- Creating workflows: To examine an app’s workflow, test cases in an E2E test suite have to be run in a particular sequence. This sequence must match the path of the end-user as they navigate through the app. Creating test suites to match this workflow can be taxing, especially since they usually involve creating and running thousands of tests.
- Accessing Test Environment: It is easy enough to test apps in dev environments. However, every application has to be tested in client or production environments. Chances are, that prod environments are not always available for testing. Even when they are, testers have to install local agents and log into virtual machines. Testers also have to prepare for and prevent issues like system updates that might interrupt test execution.The best way to access an ideal test environment is to test on a real device cloud. If in-house devices labs are not an option, use a service like BrowserStack that offers a cloud Selenium grid of 2000+ real devices and browsers for testing purposes. Access a device that users will actually use to navigate an app, and ensure that the app functions flawlessly in real user conditions.
Why is End to End Testing necessary?
Every application is connected to and integrated with multiple systems and databases outside its own environment. Needless to say, this makes the app’s workflow fairly complicated.
E2E testing determines if various dependencies of an application are working accurately. It also checks if accurate information is being communicated between multiple system components.
- Backend: E2E testing verifies the database and backend layers of an app. This is necessary since the app’s core functionalities depend on backend capabilities.
- Multi-tier system: If an application has a complex architecture whose workflow works through multiple tiers, E2E testing is necessary to verify overall functions as well as the interaction between individual tiers in the architecture.
- Distributed Environment: If an application is based on SOA (service-oriented architecture) or cloud environments, E2E testing is necessary. It is also necessary for apps consisting of multiple components that need to work in tandem for successful operation.
- Consistent User Experience: Since E2E testing involves the frontend, it also ensures that the app provides a user experience that works across multiple devices, platforms, and environments. Cross browser compatibility testing, for example, is an important part of E2E testing in this regard.
Best Practices for End to End Testing
To test end to end, it is important to follow the practices outlined below – meant to ensure smooth testing and feasible cost management.
- Prioritize the End Use: When creating test cases, test like the user. Get into the mindset of someone using the app for the first time. Is it easy to find all the options? Are the features clearly marked? Can users get what they want in two or three steps? Use acceptance testing documents and user stories that help clarify the user’s point of view. Design test cases accordingly.
Focus E2E tests on features of the app whose failure will cause maximum issues. Start with these features, and design more elaborate test cases to verify them. Consider using a risk assessment matrix for this purpose.
- Try to avoid Exception Testing: E2E Testing is best used to test common user scenarios. When it comes to exceptional user scenarios, use integration testing, or low-level unit testing.
- Maintain Order: Since E2E testing encompasses the entire application, test cases are bound to be complex. Each system component has to be tested, which increases points of failure and also the difficulty in debugging each anomaly. Structure and organization are crucial in E2E testing. Conduct unit tests and integration tests initially to resolve the first level of errors.
- Optimize setup and teardown mechanisms: Ensure that the test environment is ready to start testing at any moment. Setup processes need to be as minimal as possible. Once tests complete, it should take an equally low amount of time to erase the test data so that the environment reverts to a pristine state – thus ready to conduct tests again.
Given the importance of end to end testing, it needs to be planned from the very beginning of a project. Note that end to end testing is best conducted manually since it allows testers to put themselves in the users’ shoes. However, if automation testing is required, it is best to limit it to features that are low-risk and simply require repetitive action (For eg., filling in a signup form).
Start viewing end to end testing as a non-negotiable part of the testing pipeline. Prepare for it from the earliest stages, invest time, effort and resources in it so that the user does not have to deal with preventable disruption in their experience.