Organize Playwright tests for better reporting

Learn how to organize Playwright tests by using the project, build, and name capabilities on BrowserStack Automate.

Important: Playwright testing on BrowserStack (beta) is not available to all users yet. If you’d like to get access, please submit a request here.

You can organize your automated Playwright tests that you run on BrowserStack by ‘projects’ and ‘builds’ by using the project, build, and name capabilities while you start your Playwright test. The following table explains the different capabilities that you can specify:

Capabilities Description Expected values
name Name for your session / test case A string.
Note that only letters (A-Z, a-z), digits (0-9), periods (.), colons (:) and underscores (_) are allowed. Any other characters are ignored.
build Build version / number A string.
Note that only letters (A-Z, a-z), digits (0-9), periods (.), colons (:) and underscores (_) are allowed. Any other characters are ignored.
project Name of your project A string.
Note that only letters (A-Z, a-z), digits (0-9), periods (.), colons (:) and underscores (_) are allowed. Any other characters are ignored.

How to set the project, build and name Capabilities

For every Playwright test that you run on BrowserStack you need to connect to the BrowserStack CDP endpoint using playwright.connect. Within the CDP endpoint URL, we accept a string of capabilities at the end and you need to send all these project, build and name capabilities for your tests using this as shown below:

const { chromium } = require('playwright');

const cp = require('child_process');
const clientPlaywrightVersion = cp.execSync('npx playwright --version').toString().trim().split(' ')[1];

const caps = {
    'project': 'Your project name goes here',
    'build': 'Your specified build name goes here',  
    'name': 'Your specified test name goes here',
    'browserstack.username': process.env.BROWSERSTACK_USERNAME || 'YOUR_USERNAME',
    'browserstack.accessKey': process.env.BROWSERSTACK_ACCESS_KEY || 'YOUR_ACCESS_KEY',
    'client.playwrightVersion': clientPlaywrightVersion
  };
  const browser = await chromium.connect({
    wsEndpoint: `wss://cdp.browserstack.com/playwright?caps=${encodeURIComponent(JSON.stringify(caps))}`,
  });

As shown in the snippet above, the chromium.connect actually connects to a CDP endpoint at BrowserStack and in that URL, we accept the list of params as specified in the caps JSON object that was defined above.

Best practices for setting the different capabilities

As we have seen in the above code snippet, the project, build and name capabilities can be specified to take any value but hard coding all those values in your script is probably not a good idea.

Instead, you can follow the steps mentioned below to ensure that you get the best experience that these capabilities can provide:

  1. Define environment variables for project and build names for e.g.
    export PUPPETEER_BUILD_NAME="staging-regression-9-august"
    export PUPPETEER_PROJECT_1="E2E Regression"
    
  2. Once you have the environment variables, you can simply populate the build and project capabilities using the values as defined in the environment variables, as shown below:
    const caps = {
        'project': process.env.PLAYWRIGHT_PROJECT_1,
        'build': process.env.PLAYWRIGHT_BUILD_NAME
    }
    
  3. The advantage of using environment variables is, you can simply change the value of the environment variable before every build run on BrowserStack and you’d get newer tests organized in new build names every time (see next section to understand the benefits for using build, project and session names)
  4. You can hard code the name of each Playwright test because that will most probably not change across different build runs. Also, it is advisable to keep meaningful names of tests so that it becomes easily identifiable in the BrowserStack Dashboard.

Why should you organize your Playwright tests?

We’ve gone through the how and what part of organizing Playwright tests on BrowserStack. Now, we’ll take a look at the why part of it.

Organizing your Playwright tests by project, build and test name gives you multiple advantages when it comes to debugging your tests on the BrowserStack Dashboard. We take a look at the benefits with the help of examples:

  1. Using the project capability would let you filter out only the tests that pertain to that project, in the BrowserStack Dashboard. The following snapshot of the Automate Dashboard shows the filter on the left pane that you can use to apply the project filter.

    BrowserStack dashboard showing filter option
  2. Using the build capability will organize all test sessions with the same build name, under the same build. You can then access any session in that build simply by clicking on the build in the left pane of Automate Dashboard.
  3. Using the name capability will enable you to recognize each individual test in the Automate Dashboard by its name. You also have the status of the test on the side and hence you can quickly understand exactly which test has passed/failed.

The below snapshot of a typical Automate Dashboard shows the list of builds in the left pane. On clicking on any build name, the right side shows the list of sessions that are a part of that build (basically all sessions where the same build name was used in capability). You can also see the test name for each session and it is evident from the example that using descriptive test names is beneficial for debugging purposes later on.

BrowserStack dashboard showing filter option

Note: Learn more about how to mark tests as passed or failed on BrowserStack so that the status appears correctly on the Automate Dashboard.

We're sorry to hear that. Please share your feedback so we can do better







Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better







Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback!

Talk to an Expert
Talk to an Expert