End To End Testing: A Detailed Guide
Shreya Bose, Technical Content Writer at BrowserStack - February 5, 2020
What is End to End Testing?
End to end testing (E2E testing) refers to a software testing method that involves testing an application’s workflow from beginning to end. This method basically aims to replicate real user scenarios so that the system can be validated for integration and data integrity.
Essentially, the test goes through every operation the application can perform to test how the application communicates with hardware, network connectivity, external dependencies, databases, and other applications. Usually, E2E testing is executed after functional and system testing is complete.
Why is End to End Testing necessary?
Every application is connected to and integrated with multiple systems and databases outside its own environment. Needless to say, this makes the app’s workflow fairly complicated. E2E testing basically checks that the application works as expected at every level of integration – frontend to backend including all interfaces and endpoints.
E2E testing determines if various dependencies of an application are working accurately. It also checks if accurate information is being communicated between multiple system components.
- Backend: E2E testing verifies the database and backend layers of an app. This is necessary since the app’s core functionalities depend on backend capabilities.
- Distributed Environment: If an application is based on SOA (service-oriented architecture) or cloud environments, E2E testing is necessary. It is also necessary for apps consisting of multiple components that need to work in tandem for successful operation.
- Multi-tier system: If an application has a complex architecture whose workflow works through multiple tiers, E2E testing is necessary to verify overall functions as well as the interaction between individual tiers in the architecture.
- Interaction Accuracy: Since E2E testing involves the frontend, it also ensures that the app provides a user experience that works across multiple devices, platforms, and environments. Cross browser compatibility testing, for example, is an important part of E2E testing in this regard.
Steps for End to End Testing Process
The steps below are required to initiate and complete any E2E test.
- Analyze requirements. Have a clear idea of how the app is supposed to work in every aspect
- Set up a test environment in alignment with all the requirements
- Analyze software and hardware requirements
- List down how every system needs to respond
- List down testing methods required to test these responses. Include clear descriptions of standards (language, tools, etc.) to be followed in each test.
- Design test cases
- Run tests, study and save results
End to End Testing Example
Let’s say testers have to verify the functioning of a Gmail account. The following features have to be tested:
- Type URL into the address bar to launch the Gmail login page.
- Log into account with valid credentials.
- Access Inbox. Open Read and Unread emails.
- Compose a new email.
- Reply to and forward an existing email.
- Open Sent items folder. Check emails there.
- Open Spam folder. Check emails there.
- Log out of Gmail by clicking ‘logout’.
E2E Testing must include the following three categories of activity:
1. User Functions
To build user functions, do the following:
- List the features of the software and its interconnected sub-systems.
- For each function, track and record all actions performed. Do the same for all input and output data.
- Identify all relations between user functions.
- Establish if each user function is independent or reusable.
To build conditions based on user functions, do the following:
- Decide a set of conditions for every user function. This could include timing, data conditions, etc. – essentially any factor that can affect user functions.
3. Test Cases
To build test cases for E2E Testing, keep the following in mind:
- Create multiple test cases to test every functionality of user functions.
- Assign at least a single, separate test case to every condition.
Relevant factors to measure for End to End Testing
Consider the following factors to track the progress of end to end tests
- Test case preparation: Are the right test cases being created to test every feature of a software? The graph representing this should show the rate at which current vs planned test cases are being crafted.
- Weekly test: A percentage base report of test completion each week is extremely necessary. The important parameters here are currently executed, not executed and failed test cases.
- Defect Details: A percentage of defects that are identified, closed or opened each week is important. By distributing defects (for resolution) based on priority, it helps testers and managers keep track of bugs and supervise their resolution.
- Availability of testing environment: This provides a comparison of the amount of time that is scheduled for testing every day vs the time actually spent.
Main Challenges in End to End Testing
Naturally, detecting bugs in a complex workflow entails challenges. The two major ones are explained below:
- Building the workflow: To examine an app’s workflow, test cases in an E2E test suite have to be run in a particular sequence. This sequence must match the path of the end-user as they navigate through the app. Creating test suites to match this workflow can be taxing, especially since they usually involve creating and running thousands of tests.
- Accessing the Test Environment: It is easy enough to test apps in dev environments. However, every application has to be tested in client or production environments. Chances are, that prod environments are not always available for testing. Even when they are, testers have to install local agents and log into virtual machines. Testers also have to prepare for and prevent issues like system updates that might interrupt the execution of the test in production. The best way to access an ideal test environment is to test on a real device cloud. If in-house devices labs are not an option, use a service like BrowserStack that offers a cloud Selenium grid of 2000+ real devices and browsers for testing purposes. Access a device that users will actually use to navigate an app, and ensure that the app functions flawlessly in real user conditions.
Best Practices for End to End Testing
To test end to end, it is important to follow the practices outlined below. These techniques are meant to ensure smooth testing and efficient cost management.
- Prioritize the End Use: When creating test cases, test like the user. Get into the mindset of someone using the app for the first time. Is it easy to find all the options? Are the features clearly marked? Can users get what they want in two or three steps? Use acceptance testing documents and user stories that help clarify the user’s point of view. Design test cases accordingly.
- Try to avoid Exception Testing: E2E Testing is best used to test common user scenarios. When it comes to exceptional user scenarios, it is better to use integration testing or low-level unit testing to verify their validity.
- Analyze the risks: Focus E2E tests on features of the app whose failure will cause maximum issues. Start with these features, and design more elaborate test cases to verify them. Consider using a risk assessment matrix for this purpose.
- Maintain Order: Since E2E testing encompasses the entire application, test cases are bound to be complex. Each system component has to be tested, which increases points of failure and also the difficulty in debugging each anomaly. Structure and organization are crucial in E2E testing. Conduct unit tests and integration tests initially to resolve the first level of errors. Once end to end testing begins, continue to run smoke and sanity tests in combination with high-risk user scenarios.
- Keep a close eye on the test environment: Before testing begins, ensure that the test environment is capable of running all the suites. Document all requirements so that nothing is left unfulfilled by the system before testing starts. If possible, keep the staging environments as close to prod as possible. This ensures that not too many new bugs will pop up in production.
- Optimize setup and teardown mechanisms: Ensure that the test environment is ready to start testing at any moment. Setup processes need to be as minimal as possible. Once tests complete, it should take an equally low amount of time to erase the test data so that the environment reverts to a pristine state – thus ready to conduct tests again.
Given the importance of end to end testing, it needs to be planned from the very beginning of a project. Note that end to end testing is best conducted manually since it allows testers to put themselves in the users’ shoes. However, if automation testing is required, it is best to limit it to features that are low-risk and simply require repetitive action (For eg., filling in a signup form).
Start viewing end to end testing as a non-negotiable part of the testing pipeline. Prepare for it from the earliest stages, invest time, effort and resources in it so that the user does not have to deal with preventable disruption in their experience.