Optimize app test suite

Optimize an app test suite with Test Companion to cut App Automate sessions, minutes, and runtime without losing coverage. You approve every change first.

Test Companion reads every test in your app suite and finds work that costs sessions or minutes without adding coverage. It proposes each change in a table and waits for your approval. After you approve, it applies the changes, re-runs the tests, and opens one pull request. The pull request shows the sessions, minutes, and runtime measured before and after the changes. Test Companion does not make a change unless it can show that coverage is kept.

Prerequisites

Before you optimize an app suite, make sure you have the following:

• A supported IDE with the Test Companion extension installed.
• The test suite open in your workspace, on a branch you can open a pull request from.
• The suite configured to run on BrowserStack App Automate. Test Companion measures sessions and minutes from App Automate runs. If the suite runs only locally, Test Companion can measure runtime only and reports no session or minute numbers.
• A passing baseline on the base branch. Test Companion lists tests that already fail by name and leaves them unchanged.

Optimize an app test suite

A short prompt is enough to start. Test Companion asks for any detail it cannot infer, and it works on one folder or the whole suite.

1. Open the Test Companion panel in the IDE.

   

2. Click App at the bottom of the panel to switch to App mode.

   

3. In the chat box, describe the suite and the cost you want to reduce. You can include the following details:

   • The suite folder, attached or referenced with @.
   • The cost that matters most to you, such as sessions, minutes, or runtime.
   • Any folder or test that Test Companion must not change.

   

4. Press Enter. Test Companion reads the suite before it proposes anything.

5. Review the change table that Test Companion posts. Approve it, or ask for changes. Test Companion changes nothing until you approve.

   

6. Follow the progress in the chat. As Test Companion edits files, the changed-file row above the chat box lists them.

7. Click Keep to accept all changes or Undo to revert them. Expand the row to keep or undo one file at a time.

   

Example prompt

The following prompt names the suite, the cost that matters, and the files to leave alone:

Optimize the WebdriverIO Appium suite in @android/examples. It runs on BrowserStack App Automate using the conf.js files in each example folder. The four example folders repeat the same login and search flow, so I expect duplicated sessions. Sessions matter most to me. Keep all coverage. Show me the change table with one row per test before you change anything. Do not touch any conf.js file.

Review the change table

Test Companion classifies each test by what its code does, not by its name. It lists one row per test, including the tests it leaves unchanged, so you can see every decision. The column layout varies between runs. Each row states what the test does, the proposed change, and one of the following labels:

A test that Test Companion cannot prove is safe to change is kept, and its row says so. The table also lists larger costs that Test Companion noticed but did not change. A sign-in repeated in every test is one example. You can decide on those separately.

What Test Companion changes

Approved changes follow the conventions of your framework and reuse your existing page objects and helpers. Each kind of change has its own coverage guard:

Before it modifies any file, Test Companion creates a checkpoint of your workspace. Use Changes to review the edits, or Restore to revert them.

How Test Companion protects coverage

Optimization never overrides coverage. Test Companion applies the following rules:

• It never removes or weakens an assertion.
• It does not make a change unless it can show that coverage is kept.
• A test counts as covered by another test only at the same tier or higher. A smoke test is not covered by a nightly test.
• Every number comes from a before-and-after run, never from a file count. Merging two files into one that still runs every test saves nothing. The table says so.
• After the changes are applied, a second, separate agent reads every changed file and looks for lost coverage. Test Companion does not sign off on its own work.

Some work is out of scope. Test Companion does not add coverage or choose which tests to run for a change. It does not fix tests that already fail on the base branch, and it does not stabilize flaky tests. It reports each of these by name so you can handle them in the right flow. To fix failing tests, use Fix failed app tests.

Review the results

After each change, Test Companion re-runs only the changed tests, normally on your machine. After the last approved change, it runs the whole suite once on App Automate. The measured numbers come from that final run. Test Companion then opens one pull request that contains the following:

• The sessions, minutes, and runtime measured before and after the changes. A suite that runs only locally gets runtime only.
• Each change with its label.
• The costs it noticed but did not change.
• Any test that already failed on the base branch, by name.

Review the pull request as you would any other. If a test fails during a run because of the test itself, Test Companion fixes the test. If it fails because of an app bug, Test Companion reports the bug.

Supported frameworks

Test Companion optimizes Appium suites, including WebdriverIO and Java, and native Espresso, XCUITest, Flutter, and Maestro suites. Changes follow the conventions of the framework already in your suite.

Next steps

• Fix failed app tests: Analyze and fix app tests that fail outright, from the Failure Analysis panel or the chat.
• Automate tests: Generate automation scripts for your app.
• AI settings: Configure how Test Companion applies changes.