Validate pull requests and comments on GitHub

GitHub PR checks automatically show the status of your Test Reporting & Analytics builds directly in your pull requests. Instead of manually checking if tests passed, you’ll see pass/fail status checks to prevent broken code from being merged.

When you create a pull request, the integration performs the following tasks:

• Finds matching builds in Test Reporting & Analytics based on your commit.
• Automatically evaluates check status using your configured criteria.
• Reports results as GitHub status checks on your PR and adds rich comments for you to take additional action.

How to configure GitHub checks

Follow these steps to configure GitHub checks on Test Reporting & Analytics:

Lifecycle of PR checks

Here’s what happens when you create a pull request!

1. PR Created
   • BrowserStack Integration automatically creates checks on your Pull Request if there are any.
   • Status check appears as “Pending” in your PR.
2. Build Search
   • Integration looks for matching Test Reporting & Analytics builds.
   • Searches by project, build name, commit SHA, and timestamp.
3. Status Evaluation
   • Evaluates the build against your configured criteria (Quality Gate Status or Build Status).
   • Updates GitHub with ✅ Pass, ❌ Fail, or 🔄 Pending and adds comments.

Repository setup

Understanding how your code and tests are organized is crucial for proper configuration. Use this as a guide to ensure you indicate the right options in the check creation or configuration process.

Creating PR checks for your test repository or a monorepo

If you are raising PR checks against a test repo and using the BrowserStack SDK to run tests, Test Reporting & Analytics will automatically detect the git commit ID and append it to the build’s metadata. If you are using any other option like JUnit XML or Allure report upload, we recommend you to either use the CLI for automatic detection of Git Commit ID, or you can pass the git commit ID as part of the API’s advanced options.

• In check configuration, ensure you select that you are creating checks for your test repo/monorepo.
• Test Reporting & Analytics will then automatically match commits, with no additional setup required.

Creating PR checks for your app repository, and your tests run from a different repo

Your application code and tests are in separate repositories as follows:

my-app/            # Application repo
├── src/
└── package.json


my-app-tests/      # Test repo
├── tests/
└── package.json

Configuration:

• In the check configuration, ensure you select that you are creating checks for your app’s repo (and therefore your tests are running from a different repo).
• To successfully map your test build to your app repo’s latest commit, we recommend you pass your app repo’s commit ID as a build tag (eg. if your git commit ID is “abc123def”, ensure when you trigger your build on Test Reporting & Analytics, you set a build tag matching the commit id, i.e. “abc123def” ).
• WARNING: If you do not do the above, we will match the latest matching build and project name with a timestamp after your commit’s timestamp. For the most accurate build linking, ensure you pass the git commit ID of your app repo.

Best practices

Follow these general guidelines while using GitHub checks:

• Use descriptive check names that your team understands.
• Tag builds with environment information or release information for better filtering.
• Establish a consistent process for passing app repo commit IDs as build tags.
• Use quality gate status for more intelligent pass/fail decisions.
• Test with a few repositories before rolling out organization-wide.

Track different aspects

You can create multiple checks to track different aspects. As Test Reporting & Analytics supports multiple types of testing, including those run outside of BrowserStack infrastructure (like on your own test infra, local machine, or CI runners), you can use this to ensure a full spectrum of checks passes. Some examples are:

• “E2E Tests” - Critical user flows
• “Unit Tests” - Code coverage and unit test suite
• “API Tests” - Backend integration tests
• “Performance Tests” - Load and performance benchmarks

FAQ

Q: How quickly do checks update?
A: Checks update within 1-2 minutes of build completion in Test Reporting & Analytics.

Q: Can I use this with private repositories?
A: Yes, the GitHub App works with both public and private repositories. Just provide permission to your repository.

Q: How do I disable a check temporarily?
A: Edit the check configuration and disable it, or delete the check entirely.