BrowserStack Appium FAQs

Parallel test sharding (recommended)

• Shard your test suite: Run different subsets of your test suite simultaneously across multiple devices.
• Example: Each device handles a fraction of the load (e.g., 40 tests instead of 200), keeping session duration under safe OS limits (under 45 minutes).

Example TestNG configuration:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="iOS Regression Suite" parallel="tests" thread-count="5">
    <test name="Onboarding Tests - Device 1">
        <parameter name="device" value="iPhone 14"/>
        <classes>
            <class name="com.tests.Onboarding"/>
        </classes>
    </test>
    <test name="Payment Tests - Device 2">
        <parameter name="device" value="iPhone 13"/>
        <classes>
            <class name="com.tests.Payments"/>
        </classes>
    </test>
</suite>

Why this works: Each device only handles a subset of tests, reducing session duration and improving stability.

Optimize Appium settings

• Disable video recording if not needed to save processing power.
• Reduce log level: Set Appium log levels to info or warn instead of debug for production runs.

Problem: When running large regression suites (e.g., 200+ tests) on real iOS devices, the application may unexpectedly move to the background or terminate after 60–90 minutes of execution, even if individual tests pass in isolation.

Common symptoms:

• The app minimizes or closes without a specific Appium error code.
• Crash logs (if retrieved) show EXC_RESOURCE, wakeups, or high memory warnings.
• The device appears sluggish in video recordings towards the end of the session.

Root cause: Real iOS devices have strict OS safeguards (Watchdogs) to preserve battery and system health. Long-running automation sessions can trigger the iOS Watchdog, causing the OS to forcibly run in the background or kill the process.

Session batching (restarting the driver)

• Group your tests: Split your test list into batches (e.g., 10 groups of 20 tests).
• Lifecycle hook: Use your test runner’s @AfterClass or @AfterAll hook to tear down the driver.
• Re-initialize: Use @BeforeClass to start a fresh installation for the next batch.

Why this works: Restarting the Appium driver every 20–30 tests clears the application cache, dumps temporary logs, and resets the iOS Watchdog timer.

Symptom: Tests complete successfully on the BrowserStack infrastructure, but the local Visual Studio Test Explorer crashes during the “cleanup” or post-processing phase. The error log shows:

Unhandled exception. System.ArgumentOutOfRangeException: Index was out of range… at Newtonsoft.Json.Linq.JContainer.GetItem.

Cause: This is typically a local metadata parsing error. It occurs when the BrowserStack Test Adapter attempts to read test result files using a version of the Appium.WebDriver or .NET runtime that has conflicting JSON handling logic.

Resolution:

• Update Appium.WebDriver: Upgrade the Appium.WebDriver NuGet package to the latest stable version (e.g., v5.0.0+). This usually resolves the index mismatch.
• Check .NET Compatibility: If you are using .NET 9.0, try downgrading the project to .NET 8.0 (LTS). Some newer runtimes change how file-system watchers report data, which can trigger this exception in the Test Adapter.
• Refresh Dependencies: Ensure BrowserStack.TestAdapter and NUnit3TestAdapter are updated to their latest versions to ensure compatibility with Visual Studio 2026.

Symptom: Your obfuscated iOS app crashes on App Automate iOS 17 devices with EXC_BAD_ACCESS or KERN_PROTECTION_FAILURE, but works fine on iOS 18 and above, and works on iOS 17 without automation.

Cause: This is a known incompatibility between Promon SHIELD (and similar runtime shielders that hook vm_protect or mprotect) and the iOS 17 developer disk image (DDI) loading mechanism.

When App Automate launches your app using WebDriverAgent, iOS 17 loads XCTAutomationSupport.framework into your app’s memory and transitions its pages from read-write (RW) to read-execute (RX) after fixups. Promon SHIELD intercepts this exact vm_protect call. The page stays RW, and XCTest crashes the moment it calls into the framework.

This does not affect iOS 16, which uses a classic DDI with a different load path, or iOS 18 and above, where dyld-4 maps developer images directly as RX with no userspace transition.

Resolution:

To unblock testing immediately, request a Promon-wrapped build with anti-instrumentation relaxed for test or staging builds, and run it on App Automate iOS 17 to confirm it works.

Contact Promon SHIELD support with your crash log and request one of the following:

• Add a config option that disables anti-instrumentation for test builds.
• Allowlist RW to RX transitions for images loaded from /System/Developer/ and /private/preboot/Cryptexes/ paths on iOS 17.

Note: This is not an App Automate bug. App Automate uses the same XCTest launch path on every iOS version. The crash is caused by the interaction between Promon SHIELD’s hooks and the iOS 17 on-device loader.