Understanding Playwright Assertions

What are Playwright Assertions?

Playwright is a versatile framework designed for automating web application testing across multiple browsers. Central to its functionality are Playwright assertions. 

Playwright Assertions validate the behaviour of App Under Test. It decides whether the application is behaving as per expectations or not, based on which it assigns Fail or Pass to a Test Case.

By incorporating assertions into Playwright tests, you can confirm that the application performs as intended, enhancing the reliability and quality of the software.

Playwright provides diverse assertion types:

• Element States: Check the visibility, availability, and interactivity of UI elements.
• Content Validation: Ensure elements display the correct text, values, or match specific patterns.
• Page Properties: Assertions can confirm page details like URLs, titles, or cookie presence.
• Network Interactions: Verify the outcomes of network requests and responses to ensure proper data loading and form submissions.

Playwright Expect() Function

Playwright provides assertion using expect() function. Using expect() you can assert the expected result with actual result 

For Example:

test("Validate BrowserStack demo application site title", async ({page}) => {

   await page.goto("https://bstackdemo.com/")

   await expect(page).toHaveTitle("StackDemo")

})

In the above code example, using BrowserStack’s Demo application in the playwright test to validate the Site Title.

Code Breakdown:

Below code will access the BrowserStack’s Demo Application

await page.goto("https://bstackdemo.com/")

Below code Asserts the BrowserStack’s Website Title.

await expect(page).toHaveTitle("StackDemo")

Here using expect() function and passing the page as input. Then chaining the expect() function with matcher toHaveTitle() which accepts the string input, where you are passing the Website title. 

Different Types of Playwright Assertions

Assertions in Playwright is broadly classified into below types

• Auto-retrying Assertions
• Non-retrying Assertions
• Negating Matchers
• Soft Assertions

Let’s look into each assertion type.

Auto-retrying Assertion

Auto-retrying assertions in Playwright are a vital functionality that significantly boosts the reliability and stability of test scripts by repeatedly attempting to verify assertions until they either succeed or a predefined timeout is reached. 

Auto-retrying feature is especially useful in scenarios where web elements might not immediately meet expected conditions due to network delays, dynamic content loading, or client-side scripting operations.

Let’s write a test which asserts a text from BrowserStack’s Demo Application Website.

In this test, performing below steps:

1. Access BrowserStack’s Demo application
2. Validate the default Number of Products found

test("Validate BrowserStack default products found count", async ({page}) => {

 await page.goto("https://bstackdemo.com/")

 const productLocator = await page.locator(".products-found span")

 await expect(productLocator).toHaveText('25 Product(s) found')

})

Code Breakdown

Visit the BrowserStack’s Demo application

await page.goto("https://bstackdemo.com/")

Get the locator reference

const productLocator = await page.locator(".products-found span")

Assert the Number of Products found from Web page against expected value

await expect(productLocator).toHaveText('25 Product(s) found')

Observe that in the above assertion you pass the locator reference and then you are using matcher toHaveText which accepts expected value as string.

If you run the above test, the test will fail in assertion with auto retry timeout.

Note that the test is timed out after retrying for 5000 milliseconds, this is because Playwright has a default timeout of 5000 milliseconds.

You can overwrite the assertion timeout at command level like below.

await expect(productLocator).toHaveText('25 Product(s) found',{timeout: 2000})

Below is the full list of Auto-Retrying assertions.

Non-retrying Assertions

Non-retrying Assertions are only useful when web pages load data asynchronously. Test assertion will fail without any timeout or retrying when using the Non-retrying Assertion and below is an example test for the same.

test("example for non-retrying assertion", async ({page}) => {

 await page.goto("https://bstackdemo.com/")

 const productLocator = await page.locator(".products-found span")

 const productSearchText = await productLocator.innerText()

 await expect(productSearchText).toBe('25 Product(s) found')

})

When you run this test, it will fail with assertion without retrying.

Below is the full list of Non-retrying assertion matchers

Negating Matchers

Negating Matchers are used when we want to check that a certain condition does not hold true. It essentially reverses the condition you’re checking for, enabling you to assert the absence of a condition or element. Negating Matchers are especially helpful for ensuring that a web page or application is free from errors, incorrect states, or unwanted elements.

Below is the example test. The test will assert for filter count not matching the value 3

test("example for negating matcher", async ({page}) => {

 await page.goto("https://bstackdemo.com/")

 const filter = await page.locator(".filters .filters-available-size")

 const filterCount = await filter.count()

 await expect(filterCount).not.toEqual(3)

})

Soft Assertions

By default Assertion will abort the test as soon as the expected result is not matched with the actual result. There are cases where we have to check multiple assertions and at the end of the test throw the assertion error.

Soft assertion is good for cases where we want to assert multiple cases and then fail the test at the end.

Below is the example test. The test has two assertions and both will execute and fail.

test("example for soft assertion", async ({page}) => {

 await page.goto("https://bstackdemo.com/")

 const filter = await page.locator(".filters .filters-available-size")

 const filterCount = await filter.count()

 await expect.soft(filterCount).not.toEqual(4)

 await expect.soft(page).toHaveTitle("StackDemo!")

})

If you don’t use soft assertion, then the test will fail at the first assertion check and doesn’t continue.

Running the above test will show two assertion errors.

Playwright Custom Matchers with Examples

Playwright provides users to create their own Custom Matchers, which can be chained with expect() for Assertions

To create Custom Matchers, we need to extend the expect() function from Playwright and then add our Custom Matchers within the extend function like below

import { expect as baseExpect } from '@playwright/test';
import type { Page, Locator } from '@playwright/test';


export { test } from '@playwright/test';

export const expect = baseExpect.extend({
 async toHavePrice(locator: Locator, expected: number, options?: { timeout?: number }) {
   const assertionName = 'toHavePrice';
   let pass: boolean;
   let matcherResult: any;
   try {
     await baseExpect(locator).toHaveText(String(expected), options);
     pass = true;
   } catch (e: any) {
     matcherResult = e.matcherResult;
     pass = false;
   }

   const message = pass
     ? () => this.utils.matcherHint(assertionName, undefined, undefined, { isNot: this.isNot }) +
         '\n\n' +
         `Locator: ${locator}\n` +
         `Expected: ${this.isNot ? 'not' : ''}${this.utils.printExpected(expected)}\n` +
         (matcherResult ? `Received: ${this.utils.printReceived(matcherResult.actual)}` : '')
     : () =>  this.utils.matcherHint(assertionName, undefined, undefined, { isNot: this.isNot }) +
         '\n\n' +
         `Locator: ${locator}\n` +
         `Expected: ${this.utils.printExpected(expected)}\n` +
         (matcherResult ? `Received: ${this.utils.printReceived(matcherResult.actual)}` : '');

   return {
     message,
     pass,
     name: assertionName,
     expected,
     actual: matcherResult?.actual,
   };
 },
});

We can create a new file called fixture.ts and add the above code in that file. Once the code is added we can then write a test like below.

import { test, expect } from '../fixtures/fixtures';

test("example for custom matcher", async ({page}) => {
   await page.goto("https://bstackdemo.com/")
   const phone = await page.locator(".shelf-item__title", {hasText:"iPhone 12 Mini", })
   const phoneParent = await phone.locator("..")
   const phonePrice = phoneParent.locator(".shelf-item__price .val b")
   await expect(phonePrice).toHavePrice("699")
 })

Best Practices to use Playwright Expect()

The expect() function is a cornerstone of assertions in Playwright, offering the ability to assert on elements, network responses, and other test conditions. 

Below are the few Best Practices for Playwright Expect() :

1. Leverage Built-In Retry Mechanism

Understand the automatic retry feature in Playwright’s expect():

•  Utilize this feature for dynamic content where elements may appear or change state over time.
•  Avoid excessive reliance which might conceal performance issues or complex race conditions.

2. Employ Semantic Locators

Opt for Playwright’s advanced selectors like role and text selectors to improve both the readability and maintainability of your tests:

await expect(page.locator('text=Sign In')).toBeVisible();
await expect(page.locator('role=button', {name: 'Send'})).toBeEnabled();

These selectors enhance the semantic clarity and accessibility focus of your tests.

3. Integrate Actions with Verification

Simultaneously perform user actions and verify outcomes to mimic real user flows:

await page.click('button#save');
await expect(page.locator('text=Saved successfully')).toBeVisible();

This method validates user interactions in real-time.

4. Customize Timeouts

Adjust timeouts in expect() when the default settings do not align with specific test requirements:

• Modify the timeout parameter to suit specific waiting needs without overextending test durations.

5. Assert Non-Presence

 Assert the non-presence of elements or messages, particularly useful in validating error handling and user feedback:

await expect(page.locator('text=Error')).not.toBeVisible();

6. Utilize State-Specific Assertions

Make full use of Playwright’s state-specific assertions to directly assess the user interface:

await expect(page.locator('input[type="checkbox"]')).toBeChecked();

7. Assert Network Interactions

Capture and assert network responses to ensure backend integration is functioning as expected:

const [response] = await Promise.all([
  page.waitForResponse(resp => resp.url().includes('/api/submit') && resp.status() === 200),
  page.click('button#submit')
]);
await expect(response).toBeOK();

8. Validate Accessibility Features

Assert on accessibility features to ensure your application is accessible:

await expect(page.locator(‘role=button’, {name: ‘Confirm’})).toHaveAttribute(‘aria-live’, ‘polite’);

9. Enhance Assertion Failures

Incorporate diagnostic tools like screenshots or logs to investigate why assertions fail:

test.fail(async ({ page }) => {
  await page.screenshot({ path: 'error-snapshot.png' });
});

Why run Playwright Tests on Real Device Cloud?

Here’s why you should run Playwright tests on real browsers & devices using BrowserStack Automate:

• Diverse Environment Testing: It enables the execution of Playwright tests across a broad selection of browsers and operating systems, eliminating the necessity for maintaining local testing infrastructure. This ensures consistent application performance across various platforms.
• Concurrent Test Execution: By allowing simultaneous execution of multiple Playwright test suites, BrowserStack Automate significantly cuts down on total testing time, facilitating quicker iterative feedback and accelerated deployment cycles.
• CI/CD Integration: The platform seamlessly integrates with major continuous integration and delivery systems, including Jenkins, Travis CI, CircleCI, and GitHub Actions, automating the testing process within the development pipeline.
• Diagnostic Tools for Better Debugging: BrowserStack provides comprehensive diagnostic capabilities, including detailed logs, screenshots, and video recordings of test sessions, aiding in the swift identification and resolution of issues.
• Testing on Real Devices: Beyond simulated environments, BrowserStack also supports testing on real devices and browsers on the cloud, offering more precise and real-world test outcomes.
• Customizable Test Execution: Users can tailor test executions to meet specific needs through BrowserStack’s user interface or APIs, enabling adaptable and controlled test runs.

Talk to an Expert

Conclusion

The expect() function of Playwright is a critical asset for automation testers, delivering powerful and adaptable assertions crucial for verifying the integrity and functionality of web applications. Its inherent retry capability, coupled with its proficiency in handling complex assertions on elements, network interactions, and accessibility attributes, is particularly effective for modern web applications characterized by dynamic and asynchronous behavior.

Utilizing expect() adeptly within Playwright tests enables testers to confirm that their applications not only adhere to specific requirements but also deliver a stable user experience across diverse scenarios. This function’s ability to precisely adjust assertion conditions, such as timeouts, and to assess a broad spectrum of criteria—from the visibility of elements to the correctness of API responses—increases the precision and dependability of tests.

Running your Playwright Tests on BrowserStack’s Real Device Cloud helps you get access to 3500+ real device and browser combinations for maximum test coverage. It allows you to test under real user conditions, which will help identify the bottlenecks in the real user experience and rectify them. BrowserStack’s Automate allows you to run Playwright tests on the cloud and across browsers and devices simultaneously by leveraging parallel testing for faster testing with a vast coverage.