Playwright tests that pass in one run and fail in another, without code or application changes, are flaky tests.
They make it harder to distinguish real bugs from test instability and can lead to unnecessary reruns, wasted debugging time, and reduced confidence in CI/CD regression suites.
Timing and race conditions, shared state, external dependencies, and environment differences can contribute to flaky Playwright tests.
This guide explains how to identify these causes and prevent, detect, and fix flaky Playwright tests.
What Are Flaky Tests in Playwright?
A flaky test in Playwright is an automated test that produces inconsistent results despite no relevant change to the application or test code. The same test may pass during one test execution and fail during another.
Flaky tests create both false positives and false negatives, which waste debugging time and make it harder to catch real issues.
A flaky test fails (or passes) unpredictably even when nothing has changed; same code, same environment, same everything.
For example, a Playwright test that submits a form might pass on one run and fail on the next, with no changes in between. This is different from a test that fails under specific conditions, like a slow network.
Playwright also uses “Retries” to help identify flaky tests. A test that passes on its first run is classified as passed. A test that fails on the initial run but passes on a retry is reported as flaky. A test that continues to fail across all configured retries is reported as failed.
Why Do Playwright Tests Become Flaky?
Playwright reduces UI synchronization issues with features such as auto-waiting and resilient locators, but test design and execution conditions can still cause flakiness. The most common sources include
1. Timing and asynchronous operations
Modern applications often load elements and data asynchronously. Fixed delays can make tests unreliable when the application takes longer than expected.
await page. waitForTimeout(2000);
await page. getByRole('button', { name: 'Submit' }). click();Instead, use Playwright wait, locators, and web-first assertions that wait for the required condition:
await page. getByRole('button', { name: 'Submit' }). click();
await expect(page.getByRole('status')). toHaveText('Order submitted');Output –
2. Brittle locators and selectors
Locators based on frequently changing implementation details, such as CSS classes, can break even when the UI works correctly.
await page. locator('.btn-primary-472'). click();Prefer stable, user-facing locators such as getByRole(), getByLabel(), getByText(), and getByTestId(). This also helps Playwright’s auto-waiting and retryability.
Output –
3. Shared state and test interference
Tests can become flaky when they depend on state created or modified by another test. For example, one test may delete an account that a later test expects to use.
Keep tests independent by having each test create or arrange the state it needs. Also check for shared state across accounts, databases, files, APIs, and fixtures, especially when running tests in parallel.
4. Unpredictable test data
Random test data can make failures difficult to reproduce.
const email = `user${Math.random()}@example.com`;Prefer deterministic data where possible:
const email = 'playwright-test@example.com';
If unique data is required, use a predictable strategy and include the generated value in test output.
Output –
5. External service and network dependencies
Tests that depend on APIs, authentication systems, or payment gateways can fail because an external service is slow or unavailable.
For example, if a checkout test is validating the UI rather than the payment integration, mock the payment response instead of relying on the real gateway. Playwright’s page.route() can intercept and mock network requests.
6. Browser and environment differences
Tests can behave differently across browser engines, operating systems, viewport sizes, and network conditions. A failure that occurs consistently in one environment is more likely to be an environment-specific issue than a flaky test.
Check whether failures are random or consistently tied to a specific browser or environment before changing the test.
7. Limited resources and parallel execution
High CPU or memory usage can slow page rendering and network processing, particularly on CI machines. Similarly, parallel tests can interfere when they modify the same accounts or other shared resources.
If failures correlate with high worker counts or busy CI runners, investigate resource usage and shared state before increasing timeouts or disabling parallel execution.
Read More: Playwright Selectors: Types
How to Avoid Writing Flaky Tests in Playwright
Stable Playwright tests require specific practices that reduce flakiness. Implement the following methods with examples and code snippets:
1. Run and Debug Tests Before Committing
Execute tests locally using npx playwright test and –debug mode to catch failures early. Playwright’s debug mode pauses at breakpoints, making it easier to trace flaky behavior before pushing to CI.
Here’s how to use Playwright’s debug mode:
test('example test', async ({ page }) => {
await page.goto('https://example.com');
await page.pause(); // Opens Playwright Inspector for debugging
await expect(page.locator('h1')).toHaveText('Example Domain');
});Read More: How to start with Playwright Debugging?
2. Avoid Hard Waits
Hard waits pause test execution for a fixed amount of time, even if the app is already ready or still loading. This leads to failures when the app loads slower or faster than expected. Use Playwright’s built-in auto-waiting, which waits for elements to become visible, attached, and actionable before interacting with them.
// Avoid this
await page.waitForTimeout(5000);
// Use auto-waiting
await page.click('button:has-text("Submit")');3. Control the Testing Environment
Inconsistent environments can cause tests to behave differently across runs. Use fixed settings for viewport size, browser, and permissions to reduce variability. Set these in playwright.config.ts to ensure every test runs in the same environment:
use: {
browserName: 'chromium',
viewport: { width: 1280, height: 720 },
permissions: ['geolocation'],
}This helps prevent environment-based flakiness and makes test results more predictable.
4. Use Locators Instead of Selectors
CSS or XPath selectors often break when the UI changes. Playwright locators are more stable because they rely on roles, text, or labels. Use this approach for more reliable element targeting:
// Avoid
await page.click('#submit-btn');
// Use
await page.getByRole('button', { name: 'Submit' }).click();5. Avoid Random Values
Random inputs make tests unreliable because they produce different outcomes each run. Use fixed values or seed your randomness to keep test behavior predictable.
Here’s an example of using fixed input values:
const username = 'testuser';
await page.fill('#username', username);6. Ensure Test Isolation
Tests that depend on shared state can interfere with each other and create flakiness. Isolation tests ensure each test runs independently with a clean setup.
Here’s how to reset the state before each test:
test.beforeEach(async ({ page }) => {
await page.goto('https://example.com/reset');
});7. Configure Automatic Retry
Some failures come from temporary network or environment issues. Setting retries helps detect flaky tests by rerunning failed ones before marking them as broken.
Here’s how to configure retries in Playwright:
module.exports = {
retries: 2, // Retry failed tests up to 2 times
};Read More: How to Use Playwright Locators
8. Limit Dependence on External Resources
External services like third-party APIs can be slow or unreliable during tests. Mock these calls using page.route to avoid delays and ensure consistent responses:
await page.route('**/external-api/**', route =>
route.fulfill({ status: 200, body: JSON.stringify({ data: 'mocked' }) })
);How to Detect Flaky Tests in Playwright?
Use these methods to identify flaky tests.
1. Loop Suspected Tests to Confirm Flakiness
If a test fails intermittently in CI but not locally, it may be flaky. Run that specific test multiple times in a controlled local environment to see if it fails at random. Use a loop or a script to automate repeated execution:
for (let i = 0; i < 20; i++) {
const result = await exec('npx playwright test tests/sample.spec.ts');
console.log(`Run ${i + 1}:`, result.status);
}This helps confirm whether the issue is real or just a one-time failure.
2. Use Playwright’s Built-In Retry Feature
Retries help surface tests that fail for non-deterministic reasons. When a test passes only after a retry, it likely depends on unstable factors like timing, network conditions, or inconsistent setup.
You can enable retries globally or per test:
// Global setting in playwright.config.ts
retries: 2
// Per-test setting
test('example test', async ({ page }) => {
// test logic
}).retries(2);3. Capture Flaky Behavior With TestInfo
TestInfo in Playwright provides metadata about the current test run, such as status, retries, and error details. Logging these helps identify flaky tests by showing how often a test needs rerunning and what causes failures. For example, you can log retry attempts inside your test:
test('retry log demo', async ({ page }, testInfo) => {
if (testInfo.retry) {
console.log(`This test is retrying for the ${testInfo.retry} time`);
}
});This information highlights tests that behave inconsistently and need investigation.
4. Analyze With Trace Viewer and Video
Enable tracing or video for flaky tests to replay the test step-by-step and identify where timing or async failures happen.
use: {
trace: 'on-first-retry',
video: 'on-first-retry'
}5. Run Flaky Tests in Isolation Using Test.only
Test.only is a command in Playwright that runs only the specified test, skipping all others. Use it to isolate flaky tests that pass alone but fail when run with the full suite. This helps identify issues caused by shared state, side effects, or test order.
6. Use Strict Locators to Reduce Timing Issues
Flaky failures like “locator not found” often happen because the test tries to access elements before they appear. Using strict locators and awaiting visibility checks makes tests wait properly and reduces timing-related flakiness. Here’s how to use it:
await expect(page.locator('button:has-text("Submit")')).toBeVisible();7. Monitor flaky test patterns over time
Track recurring flaky tests using Playwright’s HTML reports or CI analytics tools. Adding tags like @flaky helps filter these tests and prioritize fixing them based on frequency and impact.
Fixing Playwright Flaky Tests
Once a flaky test is identified, fix it by targeting the root cause to ensure consistent results:
1. Analyze Failure Details
Use Playwright’s Trace Viewer to inspect failures in detail. Run the following to review test steps, DOM snapshots, and network activity.
npx playwright show-trace trace.zip
2. Use Robust Locators
Replace fragile selectors with Playwright’s role-based or chained locators for stable element targeting. For example:
await page.getByRole('button', { name: 'Submit' }).click();3. Avoid Hard waits
Remove fixed delays. Instead, wait for specific app conditions like responses or element states:
await expect(page.locator('.toast-message')).toHaveText('Saved successfully');Also Read: Understanding Playwright waitforloadstate
4. Handle Async Flows Properly
Ensure navigation and API calls are complete before proceeding. Use Playwright’s built-in waiting and assertions.
5. Run Dependent Tests Sequentially
If tests share state or data, disable parallelism for that scope:
test.describe.configure({ mode: 'serial' });6. Validate Fixes Across Environments
Test your changes locally, in CI, and across browsers to confirm stability before merging.
Playwright Flaky Test Troubleshooting: Symptom vs. Cause
A flaky test symptom is the observable, inconsistent failure that appears in your CI/CD pipeline (like a timeout or a missing/incorrect element).
The root cause is a deeper issue (API incompatibility, shared test data, or unreliable selector) that leads to its unpredictable behavior.
| Symptom | Possible cause | What to investigate |
|---|---|---|
| The element intermittently cannot be found. | Timing or locator instability | Locator strategy and application state |
| The test passes after a wait. ForTimeout() is added | Synchronization problem | Replace fixed delay with condition-based waiting. |
| Test passes alone but fails in the suite | Shared state | Test isolation and cleanup |
| The test fails only during parallel execution. | Shared resource or state conflict | Accounts, files, records, APIs |
| The test passes locally but fails in CI. | Environmental difference | Browser, OS, CPU, memory, network |
| The test fails only in one browser | Browser-specific behavior | Browser compatibility |
| The test fails initially but passes after a retry. | Intermittent condition | Timing, data, network, state |
| The test produces different errors on different runs. | Non-deterministic behavior | Data, state, dependencies, environment |
| The test becomes slower before failing. | Resource or network issue | CPU, memory, backend response time |
| Failure disappears after increasing the timeout. | A timing issue may be present. | Identify why the expected condition is delayed. |
You can use traces, screenshots, logs, and artifacts to audit your test cases and carry on your root cause analysis operations.
Read More: How to Handle Trace Viewer in Playwright?
Best Practices For Maintaining Scalable Playwright Tests
Use the following practices to keep Playwright tests reliable over time:
- Choose resilient locators that represent stable user-facing behavior, such as roles, labels, or text, so tests don’t break when UI structure changes.
- Avoid arbitrary waits and instead synchronize with application conditions using Playwright’s auto-waiting and assertions for reliable timing.
- Keep test data deterministic or reproducible so failures can be consistently recreated and debugged without randomness affecting outcomes.
- Isolate test state so each test sets up and cleans up its own data, ensuring no dependency on execution order or other tests.
- Control external dependencies when they are not the subject of the test by mocking or stubbing APIs to avoid network or service instability.
- Design tests for parallel execution without shared mutable resources to prevent race conditions and cross-test interference.
- Keep important environment settings consistent, like browser size, timeouts, and configurations, to avoid environment-driven inconsistencies.
- Use retries to identify intermittent failures, not permanently hide them, since retries only signal flakiness rather than fix root causes.
- Capture traces and other failure artifacts like screenshots, logs, and videos to understand exactly what happened during a failure.
- Validate fixes repeatedly and across relevant environments to ensure the issue is truly resolved and not just temporarily masked.
- Monitor recurring flaky tests instead of allowing them to become normal pipeline noise so they can be fixed before eroding trust in CI results
Conclusion
A flaky test is more than an intermittent failure. It is a test whose result cannot be consistently trusted.
Reduce flakiness by using resilient locators, synchronizing with application states, keeping test data deterministic, isolating tests, and controlling external dependencies.
When a test behaves inconsistently, reproduce and isolate it, then use traces and execution artifacts to identify the root cause. BrowserStack Automate can help validate Playwright tests across browsers, real devices, and environments, making it easier to detect and debug environment-specific failures.


