Test Executions widget in custom dashboards

Track how many automated tests you run over time, and compare the trend across teams, modules, and platforms.

The Test Executions widget is a line chart of test execution counts over time. It counts every execution, not unique test cases, so a test that ran ten times adds ten to the count. Use it to check whether your automation volume is growing, and to compare teams, modules, or platforms on one chart.

Summary of widget data and visualization

Each line on the chart is one segment. A segment is a named set of filters, such as one team’s projects or one platform. The legend names each segment and gives it a color. The widget gallery lists this widget as Tests Executions.

The X-axis follows the dashboard date range, so the range you pick sets how much time each point covers. A 7D range plots one point per day. A wider range, such as 3M, groups executions into weeks, so a single point covers a longer period.

Key metric information and visualization

The following table describes each element of the chart:

Add the widget

Follow these steps to add the widget to a custom dashboard:

1. Open a dashboard and select Tests Executions from the widget gallery. The widget dialog opens with a live preview.

   

2. Enter a Widget name. To add a note for readers, click Add a description and enter up to 200 characters.
3. Turn on the Show chart summary toggle to show the total count for the chart.
4. Under Segments, enter a Segment name and add the filters for that line. Click Add segment for each extra line you want to compare.
5. Check the preview, then click Save.

The following table describes each option in the widget dialog:

Segments

Segments are how you compare teams, modules, or platforms inside a single widget. Each segment is one line on the chart, and each carries its own filters. Every segment needs a Segment name, because that name labels the line in the legend.

Use the following controls on a segment card:

• Click Add segment to add a line.
• Click the copy icon to duplicate a segment with its filters.
• Click the delete icon to remove a segment. The icon appears only when the widget has more than one segment.

Available filters

Filters apply per segment, so two lines on the same chart can cover different data. No filter applies until you add one. To add a filter, click Add filter, select a field, then pick the values you want. After you add the first filter, Clear filters appears next to Filters. Click it to remove every filter on that segment.

Some filters have a match operator next to the filter name. It sets how the values you select combine. The default, Matches any one value (OR), counts an execution when it matches any selected value. When a filter also offers Matches all values (AND), that option counts an execution only when it matches every selected value.

To reuse a filter set from another widget on the same dashboard, click Import from other widgets. Apply the following filters to a segment:

The list also carries a Custom fields group for the custom fields defined in your projects.

Users and Last executed by answer different questions. Users counts the tests in the builds that the selected users triggered. Last executed by counts the tests whose current result those users set last. The user who triggered the build is the last executor until someone updates that test’s result by hand.

Drill down for more information

The chart tells you how many tests ran. The breakdown pane tells you which projects those executions belong to. Follow these steps:

1. On the dashboard, hover over a point on the chart.
2. Click View breakdown. The Tests Executions Drilldown pane opens.
3. Read the segment name, the execution count, and the period at the top of the pane.
4. Read the Project table. Each row is one project with its execution count for that period, sorted from the highest count down.
5. Click View tests next to a project. Tests Health for that project opens in a new tab, filtered to the same period.

Interpreting test data

A rising line can mean wider coverage. It can also mean the same tests running more often. Use the following patterns to tell the two apart:

Next steps

• Test executions covers the same widget on a Test Management dashboard, where you can also count manual runs.
• Unique Test Cases widget separates growth in coverage from growth in volume.
• Segments in custom dashboards covers how to build and name the lines on this chart.
• Drill down on dashboard widgets covers the breakdown pane in more detail.
• List of widgets in custom dashboards lists every widget you can add.