BrowserStack Appium FAQs

Parallel test sharding (recommended)

• Shard your test suite: Run different subsets of your test suite simultaneously across multiple devices.
• Example: Each device handles a fraction of the load (e.g., 40 tests instead of 200), keeping session duration under safe OS limits (under 45 minutes).

Example TestNG configuration:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="iOS Regression Suite" parallel="tests" thread-count="5">
    <test name="Onboarding Tests - Device 1">
        <parameter name="device" value="iPhone 14"/>
        <classes>
            <class name="com.tests.Onboarding"/>
        </classes>
    </test>
    <test name="Payment Tests - Device 2">
        <parameter name="device" value="iPhone 13"/>
        <classes>
            <class name="com.tests.Payments"/>
        </classes>
    </test>
</suite>

Why this works: Each device only handles a subset of tests, reducing session duration and improving stability.

Optimize Appium settings

• Disable video recording if not needed to save processing power.
• Reduce log level: Set Appium log levels to info or warn instead of debug for production runs.

Problem: When running large regression suites (e.g., 200+ tests) on real iOS devices, the application may unexpectedly move to the background or terminate after 60–90 minutes of execution, even if individual tests pass in isolation.

Common symptoms:

• The app minimizes or closes without a specific Appium error code.
• Crash logs (if retrieved) show EXC_RESOURCE, wakeups, or high memory warnings.
• The device appears sluggish in video recordings towards the end of the session.

Root cause: Real iOS devices have strict OS safeguards (Watchdogs) to preserve battery and system health. Long-running automation sessions can trigger the iOS Watchdog, causing the OS to forcibly run in the background or kill the process.

Session batching (restarting the driver)

• Group your tests: Split your test list into batches (e.g., 10 groups of 20 tests).
• Lifecycle hook: Use your test runner’s @AfterClass or @AfterAll hook to tear down the driver.
• Re-initialize: Use @BeforeClass to start a fresh installation for the next batch.

Why this works: Restarting the Appium driver every 20–30 tests clears the application cache, dumps temporary logs, and resets the iOS Watchdog timer.

Symptom: Tests complete successfully on the BrowserStack infrastructure, but the local Visual Studio Test Explorer crashes during the “cleanup” or post-processing phase. The error log shows:

Unhandled exception. System.ArgumentOutOfRangeException: Index was out of range… at Newtonsoft.Json.Linq.JContainer.GetItem.

Cause: This is typically a local metadata parsing error. It occurs when the BrowserStack Test Adapter attempts to read test result files using a version of the Appium.WebDriver or .NET runtime that has conflicting JSON handling logic.

Resolution:

• Update Appium.WebDriver: Upgrade the Appium.WebDriver NuGet package to the latest stable version (e.g., v5.0.0+). This usually resolves the index mismatch.
• Check .NET Compatibility: If you are using .NET 9.0, try downgrading the project to .NET 8.0 (LTS). Some newer runtimes change how file-system watchers report data, which can trigger this exception in the Test Adapter.
• Refresh Dependencies: Ensure BrowserStack.TestAdapter and NUnit3TestAdapter are updated to their latest versions to ensure compatibility with Visual Studio 2026.