Run Monitoring

Monitor the live status of different builds or understand the release readiness of your projects.

Typically when tracking the status of a product or release, there are multiple build runs (and types of build runs) that teams run or watch to ensure everything is green. This could include E2E tests, API tests, Unit tests, or even the same group of tests running across multiple geography microsites. All of these different builds need to pass to ensure your product is meeting quality standards, or your release is good to go.

The Run Monitoring Widget helps you achieve exactly this, by displaying the status of one or more builds that you can configure and monitor in one simple widget. The widget gives you a quick “go” or “no-go” indication for each build and an overall status for the entire release. In a single click, you can navigate to any build listed and take action if needed.

Widget configuration - Run Monitoring

You can configure the following options in the Run Monitoring Widget:

Widget details

• Widget name: Add a suitable name to easily identify the purpose of the widget. We recommend choosing a name like “ - Release Readiness” or “ - Live Monitoring”.
• Description: An optional widget description to explain what builds this is tracking, and any other helpful information that viewers can leverage. Users can view this description by hovering over the info icon on the widget.
• Visualization Options: Choose between Tile or Table visualization formats for your widget. The Tile visualization is the default option, offering a more visual approach to seeing the status of different builds. The Table visualization is potentially more familiar to what others users may have already used, like the Build Runs page.

Monitoring criteria

The Run Monitoring widget lets you choose from two criteria to decide if a build (and therefore a project) is good to go or not.

Choose what determines if a build is ready for release:

• Quality Gate Status (Recommended): Uses the Quality Gate status to determine if a build is good to go or not. Quality Gate offers an extremely powerful way to automatically audit a build.
• Build Status: Uses the build’s pass/fail status to determine build readiness.

All builds should pass (either in QG Status, or Build Status) depending on the option selected for the widget to call out “All good”, otherwise the widget flags the project or release as “Needs Attention”.

Targeting different build runs

The Run Monitoring widget offers a few different ways for you to target different build runs and tracked them.

Targeting a specific test run

Follow these steps to target a specific test run in the Run Monitoring widget:

• Configure the widget to track specific test runs based on your filters.
• You can create multiple ‘segments’ - which are groups of rules to identify different sets of builds. This is very to Segments in other dashboard widgets, except instead of trend lines, it adds builds to the widget.
• You can add different filters to identify the right build. If you add “Build Name”, in case the build is missing or was not run - the widget mentions this too. The timeframe configured in the top of the dashboard applies to this method of targeting builds.

Targeting build runs using creation or modification time

Follow these steps to target build runs using creation or modification time:

• Configure the widget to track runs based on when they were created or modified, with options for Last 24H, Last 1 Week, Last 2 Weeks, Last 1 Month, or Custom Date range.
• You can add different filters to identify the right build. If you add “Build Name”, in case the build is missing or was not run - the widget mentions this too.

Monitoring logic

The Run Monitoring Widget uses the following logic to determine build status:

Monitoring logic for automation runs

• Uses either Build Status or Quality Gate Status (as configured in monitoring criteria).
• Shows the latest build matching the applied filters within the specified timeframe.

Overall status

• All builds being monitored must be passing for the overall widget to show an “All Good” status.
• If any monitored build is not ready, the widget shows a “Needs Attention” status.

Sample use cases

The run monitoring widget can be configured for many useful scenarios. Some of the most common use cases are as follows:

Monitor release readiness across multiple products

You can configure separate segments for different products or components that make up your release. This gives you a single dashboard to monitor the readiness of your entire product suite before release.

Track core features and optional features separately

Configure segments to track builds for core features separately from optional features. This allows you to make informed decisions about proceeding with a release even if some optional features aren’t ready.

Monitor platform compatibility

Track build status across different platforms (Windows, Mac, mobile devices) to ensure your release is ready for all target environments.

Track release readiness by team

Configure segments for builds managed by different teams to get a quick view of which teams’ components are ready for release and which need more attention.