View Flutter test results
The test status corresponds to the outcomes and evaluations of the tests that have been run, reflecting if each test case has passed or encountered issues. A build comprises all test cases. The status of a build depends on the status of tests that are executed within the build.
As you initiate the test execution using the get build status REST API with the unique build identifier assigned to this build, you can start viewing the status of the test execution using on the App Automate dashboard. For information on the build ID, refer to the builds REST API.
Accessing test execution results
You can access the test execution results using one of the two following ways.
Once you start the Flutter test execution using REST API, you can access the test execution results on the App Automate dashboard.
Analysis of test execution
Selecting test builds: In the App Automate dashboard, you can choose a specific build linked to your test execution. Each build corresponds to a set of test cases.
Summarizing test execution: After selecting a build, you will receive a comprehensive summary of your entire test execution. This summary provides an overview of the results and performance metrics.
Reviewing individual test case results: For a more in-depth analysis of your test execution, you can examine the results for each individual test case within the chosen build. This enables you to focus on specific areas of interest.
Exploring execution details: For every individual test case, you can delve into the following execution details and debugging information:
You can obtain a comprehensive summary of your build’s progress, along with results for individual test cases within the build using the REST API.
- To access the overall build summary, use the get build status REST API. Each build is assigned a unique build identifier (ID). The build summary includes the overall status of the build and the status of each test case in the build.
Here is an example request view the build status and results using the REST API.
Below is the response to the above sample request. Note that the response contains the status of the build along with each session’s results.
{
"id": "bb3456f92211eb35a1c7527530ede1c250af2eca",
"framework": "flutter integration tests",
"duration": 65,
"status": "passed",
"input_capabilities": {
"devices": [
"Samsung Galaxy S9 Plus-9.0"
],
"app": "bs://0b5948a88c927fee4db8f07bd964e0f2030d3506",
"testSuite": "bs://5f87c81caea6ce89ac453c77eb70d75e8e157f56",
"project": "Untitled Flutter integration tests Project"
},
"start_time": "2023-08-23 09:43:09 UTC",
"app_dir_details": {
"url": "bs://5f87c81caea6ce89ac453c77eb70d75e8e157f56",
"bundle_id": "com.example.apurva_get.test",
"version": "",
"name": "flutter_testsuite.apk"
},
"devices": [
{
"device": "Samsung Galaxy S9 Plus",
"os": "android",
"os_version": "9.0",
"sharding": false,
"sessions": [
{
"id": "d088873cf1a7582c37f50e4e892505a994258732",
"status": "passed",
"start_time": "2023-08-23 09:43:47 +0000",
"duration": 27,
"testcases": {
"count": 1,
"status": {
"passed": 1,
"failed": 0,
"skipped": 0,
"timedout": 0,
"error": 0,
"running": 0,
"queued": 0
}
}
}
]
}
]
}
- To retrieve results for a specific test case within the build or its sessions, use the get sessions details REST API. Every testing session possesses a unique session identifier (ID). The session status contains detailed information about the session, such as the session’s status, the status of each test case within the session, and debugging information of each test case. You can get the session ID from the response above.
Here is an example request to obtain the session details using the REST API:
Below is a sample response of the above request:
{
"id": "d088873cf1a7582c37f50e4e892505a994258732",
"status": "passed",
"start_time": "2023-08-23 09:43:47 +0000",
"duration": 27,
"testcases": {
"count": 1,
"status": {
"passed": 1,
"failed": 0,
"skipped": 0,
"timedout": 0,
"error": 0,
"running": 0,
"queued": 0
},
"data": [
{
"class": "MainActivityTest",
"testcases": [
{
"name": "VERIFY JSON URL",
"start_time": null,
"status": "passed",
"duration": null,
"video": "https://api.browserstack.com/app-automate/flutter-integration-tests/builds/bb3456f92211eb35a1c7527530ede1c250af2eca/sessions/tests/d088873cf1a7582c37f50e4e892505a994258732df9d19cd/video#t=0,8",
"id": "d088873cf1a7582c37f50e4e892505a994258732df9d19cd",
"instrumentation_log": "https://api.browserstack.com/app-automate/flutter-integration-tests/builds/bb3456f92211eb35a1c7527530ede1c250af2eca/sessions/tests/d088873cf1a7582c37f50e4e892505a994258732df9d19cd/instrumentationlogs"
}
]
}
]
}
}
Interpreting test status and build status
The status of the executed test cases determine the overall status of the build. Each test case within the test suite and the build itself can have one of the following statuses:
BrowserStack assigns the build status shown in the table below, in order of precedence.
| Status | Test status | Build status |
|---|---|---|
Running |
A test case is being executed by the test runner. | The status of one or more tests is running. |
Failed |
The test case explicitly asserts and marks the test as failed. This can also occur if an uncaught runtime exception happens during test execution. | One or more tests has failed assertions or uncaught run-time exceptions. |
Error |
An errored test is one that failed due to an unanticipated issue on BrowserStack’s testing infrastructure. | One or more tests had errors due to unexpected BrowserStack issues. |
Timed out |
Execution of a test case (in a running state) halts due to BrowserStack timing out the session after exceeding a 2-hour limit or the test case being “idle” for 15 minutes (defined as no update from the test runner). | The status of one or more tests is timed out. |
Passed |
The test case explicitly asserts and marks the test as passed. | The status of all tests is passed (excluding skipped). |
Queued |
A test case is queued for execution. This is the default initial state. | The status of all tests is queued. |
Skipped |
A test case is never invoked during the test-suite execution, possibly due to BrowserStack timing out the session after exceeding a 2-hour limit, marking all remaining test cases as skipped. | The status of all tests is skipped. |
Related topics
-
Manage your uploaded apps using our REST API.
-
Use the recent apps API endpoint to list your recently uploaded apps.
-
Delete your uploaded apps using the delete app API endpoint.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Is this page helping you?
Yes
No
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!
