Skip to main content

Integrate your test suite with BrowserStack

This section will help you setup and integrate your test suite to run on BrowserStack App Automate. It covers the key features and best practice recommendations for seamless integration.

Setup remote Appium WebDriver

Set environment variables for BrowserStack credentials

​​BrowserStack Username and Access Key are required to initialize a remote Appium WebDriver.

In the getting started section, we set up BrowserStack credentials directly in the test script. That method works for a sample build, but for a production-grade integration we recommend you store your credentials as environment variables and use those environment variables in your test script.

# Set these values in your ~/.zprofile (zsh) or ~/.profile (bash)
export BROWSERSTACK_USERNAME="YOUR_USERNAME"
export BROWSERSTACK_ACCESS_KEY="YOUR_ACCESS_KEY"
# setx.exe does not set the environment variable in the current command prompt, but it will be available in subsequent command prompts
setx BROWSERSTACK_USERNAME "YOUR_USERNAME"
setx BROWSERSTACK_ACCESS_KEY "YOUR_ACCESS_KEY"

# Verify whether the variables have been set
echo BROWSERSTACK_USERNAME
echo BROWSERSTACK_ACCESS_KEY

Update your test configuration

Use the userName and accessKey variables to set your BrowserStack Username and Access key via environment variables in your test script file.

desired_cap = {
    "bstack:options": {
        # Set your access credentials
        "userName" : os.environ["BROWSERSTACK_USERNAME"],
        "accessKey" : os.environ["BROWSERSTACK_ACCESS_KEY"],
    }
}
import os
desired_cap = {
    # Set your access credentials
    "browserstack.user" : os.environ["BROWSERSTACK_USERNAME"],
    "browserstack.key" : os.environ["BROWSERSTACK_ACCESS_KEY"],
    #..
}

Initialize remote Appium Webdriver

You need to initialize an instance of Appium webdriver to run remote test sessions on BrowserStack’s real device cloud.

Create a new instance of Appium webdriver using BrowserStack credentials and the remote BrowserStack hub URL as shown in the following code snippet.

browserstack_sample.py

Upload your app

Upload app on BrowserStack

Use the following option to upload an Android app (.apk or .aab file) or iOS app (.ipa file) to BrowserStack servers.

  • Upload_via_REST_API Upload via REST API

Use our REST API to upload an Android app (.apk or .aab file) or an iOS app ( .ipa file) from your local filesystem. Use the command-line tool cURL to upload your app as shown below :

Copy icon
Note: On Windows, follow instructions here to install and setup cURL utility

The generated app_url is a unique ID used to identify the uploaded app build. Take note of this app_url as it will be used in steps ahead.

Copy icon
Copy icon
Copy icon
Copy icon

Specify application under test

After the app is uploaded, update the app capability in your test file with the app_url value obtained in the previous step. The uploaded app will be installed on the device during test execution.

Capability Expected values
app Your application under test. Set to app_url value, bs://<app-id> in the API response.
browserstack_sample.py

Additional information

Here’s some information that you may find useful for integration with BrowserStack.

Prepare an Android app for testing on BrowserStack

  1. For Android apps, the supported file formats are .apk and .aab.
  2. If you upload an unsigned version of an Android app, we sign them with our certificates before installing them to our devices. Similarly, whenever .aab files are uploaded, we convert them to a universal APK and sign them with our certificates.
  3. When Browserstack re-signs the apps, your app’s integration with third-party libraries, such as Google Firebase services, Google Maps SDK, Facebook SDK, etc may not work if you have restricted the use of API keys based on the SHA-1 certificate fingerprint of the app’s signing key. To avoid this issue, you can sign the APK with your own certificates before uploading the app to BrowserStack.

Prepare an iOS app for testing on BrowserStack

  1. For iOS apps, the supported file format is .ipa. Ensure that your app is built for running on real devices and not simulators.
  2. When you upload an iOS app, we will re-sign the app with our own provisioning profile to be able to install your app on our devices during test execution. Because we re-sign the application before running a session with our certificate, app entitlements are removed and if the apps are relying on these entitlements, the app might crash or app functionality will stop working.

However, if your app is signed using the Apple Developer Enterprise Program, you can disable this behavior to test features, such as push notifications on BrowserStack devices. To disable re-signing, set the browserstack.resignApp capability to false in your Appium test scripts.

Capability Description Value
resignApp for W3C protocol
browserstack.resignApp for JSON Wire protocol
Disable re-signing of Enterprise signed app uploaded on BrowserStack true, false
Default: false
desired_cap = {
  'bstack:options' {
    #...
    "resignApp": "false"
    #...
  }
}
desired_cap = {
  #...
	"browserstack.resignApp": "false"
  #...
}

Manage uploaded apps on BrowserStack

You can manage your uploaded apps using our REST API.

To list your recently uploaded apps, you can use the following options:

You can delete your uploaded apps using the delete app API endpoint. By default, we will delete the uploaded app after 30 days from the date of upload.

Test on development or staging environments

Local Testing is a BrowserStack feature that helps you test mobile apps that access resources hosted in development or testing environments during automated test execution.

You can setup localhost testing using the following two ways:

  • Via PIP (language bindings): The BrowserStack Local binary is installed using the pip command. It internally establishes a secure tunnel with BrowserStack when you run your test case. The package provides hooks that let you control the lifecycle of the tunnel. Using language bindings is the recommended way to integrate localhost testing on Browserstack.
  • Via CLI Interface (binary): You can also download the BrowserStack Local binary and setup the tunnel via your terminal. This method is useful when you don’t want to make any code changes in your test suite and want to manage the tunnel lifecycle independently.

Install the package

Add browserstack-local to your project using the pip command.

Set the access key and use available methods in your test script

Set your access key that is used to create the secure tunnel. Use the following methods provided by the local library to manage your local connection:

Method Description
bs_local.start(**bs_local_args) Expects **bs_local_args object. Returns a callback when the tunnel has started successfully. Your test script should start executing after this callback has been invoked.
bs_local.stop() Call this method after your test suite is complete.
browserstack_sample_local.py

Add desired capabilities to enable local

Update your Python test (browserstack_sample_local.py) file to set the local capability to true. You may face errors running your test script if any other capability is enabled before setting up local.

desired_cap = {
    "bstack:options": {
        #Set the local capability as true
        "local" : "true"
    }
}
desired_caps = {
    #Set BrowserStack Local capability as True
    "browserstack.local": "true"
}

If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Check out Local Binary parameters to learn about additional arguments.

Execute build on BrowserStack

You are now ready to execute your build on BrowserStack. On your local machine, open the terminal/command prompt. Navigate to either the android or ios directory and run the test script.

Download BrowserStack Local

Show download options

Unzip the binary

Unzip the downloaded file and move it to a folder/directory from which you have permission to start it using your command line or terminal.

Run the binary using your command line or terminal

Run the following command to initiate the BrowserStack Local connection

If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Check out Local Binary parameters to learn about additional arguments.

# Step 3 - Run this command in your terminal to start the BrowserStack Local binary. Your working directory should be where you have the downloaded binary.
./BrowserStackLocal --key YOUR_ACCESS_KEY
# Step 3 - Run this command in your command prompt. Your working directory should be where you have unzipped BrowserStackLocal.exe
BrowserStackLocal.exe --key YOUR_ACCESS_KEY

Set up config to enable local

Copy the capabilities from the adjacent window into your test file.

desired_cap = {
    "bstack:options": {
        #Set the local capability as true
        "local" : "true"
    }
}
desired_caps = {
    #Set BrowserStack Local capability as True
    "browserstack.local": "true"
}

Execute build on BrowserStack

You are now ready to execute your build on BrowserStack. On your local machine, open the terminal/command prompt. Navigate to either the android or ios directory and run the test script.

Migrate your test cases

This section will help you with all the configuration changes, commonly used features, and best practices for a smooth migration of your test cases to BrowserStack.

Configure OS and Device Combinations for your tests

BrowserStack App Automate offers an extensive coverage of real iOS and Android devices for automated testing. Check out the list of devices and OS versions currently supported by BrowserStack.

We recommend running your build using a single Android or iOS device to begin with. This step helps isolate issues during the migration phase and aids faster debugging. Once you’ve migrated your test cases or have achieved stability with a single device, you can try running the tests on multiple devices.

To integrate your test suite, edit the test file to set the deviceName and platformVersion capabilities to the required devices and OS version.

Capability Expected values
deviceName Set to the device you want to test on.
platformVersion Set to the OS version of the device.
desired_caps = {

    # Specify deviceName and platformVersion for testing
    "deviceName" : "Google Pixel 3",
    "platformVersion" : "9.0",

    #..
}
desired_caps = {

    # Specify deviceName and platformVersion for testing
    "deviceName" : "iPhone 11 Pro",
    "platformVersion" : "13",

    #..
}

You can also pass regular expressions (regex) in deviceName and platformVersion capabilities if your device selection is more flexible. Check out how to use regular expressions to specify device attributes.

Organize tests

Naming your tests and builds properly is crucial for effective debugging, test reporting, and analyzing your build execution time. Following are the capabilities you can use to organize your test runs.

Capability Description
sessionName Name of your test case. For example, Login Screen - Verify Login Button.
buildName CI/CD job or build name. For example, Android build #23, ios_staging_1.3.27.
projectName Name of your project. For example, BrowserStack Android App, iOS Consumer App.
browserstack_sample.py

We recommend that you use a new build name every time you run your test cases. Using a new build name upon every test run makes it easier to find and debug tests on the App Automate dashboard. Using a static build name across multiple test runs will group those tests into a single build on the dashboard.

A build can only have a maximum of 1000 tests and when the limit is reached, a new build gets created with a '-1' suffixed to the original build name, for example, alpha_0.1.7-1.

Mark test as passed or failed

Use JavascriptExecutor in your test script or use REST API to mark your test status as passed or failed on BrowserStack.

  • Using Javascript custom executor: Your test script might contain many assertions and you may choose to mark the status of the test based on any or all assertions. Based on your test logic, use a Javascript custom executor in your test case to mark the test status as pass or fail.

  • Using REST API: You can also mark a test session’s status as passed or failed using our REST API. This method is particularly useful when a test successfully completes on BrowserStack with a Completed status, but a failed assertion is seen in the test case. Use the sample cURL request and update it with your session ID to update the status and reason for that session ID.

driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'

Set up debugging capabilities

BrowserStack provides a set of debugging capabilities for your tests, which are enabled by default. You can also enable following optional capabilities to further debug your tests:

  • Visual Logs: Use the visual logs capability to capture screenshots at every Appium command automatically. Enable visual logs using the debug capability as shown in the adjacent snippet.
  • Network Logs: Network Logs capture the browser’s performance data, such as network traffic, latency, HTTP requests, and responses in a HAR format. You can enable network logs using the networkLogs capability as shown in the following code snippet.
browserstack_sample.py

BrowserStack App Automate supports multiple other debugging capabilities including video recording, device logs, crash logs and app profiling. Check out documentation to explore all available debug options.

Commonly used features and advanced use cases

Here’s a list of features and capabilities you may find useful for your tests.

Inspect elements with Appium Desktop

You can inspect your app elements and interact with them on BrowserStack’s real devices using Appium Desktop. BrowserStack’s integration with Appium Desktop allows you to try the inspector on real Android and iOS devices. Download the latest release of Appium Desktop from the Appium Desktop Github page and checkout detailed documentation for Appium Desktop integration.

Change device orientation

You can change the screen orientation of an Android or iOS device to set your application in portrait or landscape mode during the execution of the test. Use the deviceOrientation capability to change the screen orientation.

Capability Description Expected values
deviceOrientation Set the screen orientation of mobile device. portrait, landscape
Default: portrait    
browserstack_sample.py

Handle permission pop-ups

While testing different scenarios, it’s common for iOS or Android apps to show various pop-ups or system dialogs that prompt the user to grant various permissions, such as contacts, notifications, photos, etc., to the app.
Following are the ways to handle these pop-ups during automated test execution for Android and iOS apps:

  • Handle pop-ups for Android apps: Use Appium’s autoGrantPermissions capability to automatically determine which permissions your app requires and grant them to the app on install. These permissions are determined based on the Android manifest defined in the the app’s .apk file. This cabability works on Appium 1.9.1 and above.

  • Handle pop-ups for iOS apps: Use Appium’s autoAcceptAlerts and autoDismissAlerts capabilities to handle app permissions. autoAcceptAlerts will automatically accept all permission pop-ups and autoDismissAlerts will automatically dismiss all permission pop-ups. This includes privacy access permission pop-ups, such as location, contacts, and photos.

desired_cap = {
  'bstack:options': {
    "autoGrantPermissions": "true",
  }
}
desired_cap = {
  'bstack:options': {
    "autoAcceptAlerts": "true", #to accept all alerts
  }
  #OR
  'bstack:options': {
    "autoDismissAlerts": "true" #to dismiss all alerts
  }
}

Simulate GPS location

GPS Geolocation testing enables you to simulate the location of the device to specific GPS coordinates to test location-based scenarios for your app.

Capability Description Value
gpsLocation Simulate the location of the device to a specific GPS location. Latitude followed by longitude

Acceptable range for latitude is -90 to +90 and for longitude is -180 to +180.

For example, “40.730610,-73.935242”
browserstack_sample.py

Check out all BrowserStack custom capabilities

Use our capabilities builder to configure your Appium test suite.

The capability builder lets you select from a comprehensive set of options that you can use to customize your tests on BrowserStack App Automate.

Configure OS and Device Combinations for your tests

BrowserStack App Automate offers an extensive coverage of real iOS and Android devices for automated testing. Check out the list of devices and OS versions currently supported by BrowserStack.

We recommend running your build using a single Android or iOS device to begin with. This step helps isolate issues during the migration phase and aids faster debugging. Once you’ve migrated your test cases or have achieved stability with a single device, you can try running the tests on multiple devices.

To integrate your test suite, edit the test file to set the device and os_version capabilities to the required devices and OS version.

Capability Expected values
device Set to the device you want to test on.
os_version Set to the OS version of the device.
desired_caps = {
  
    # Specify device and os_version for testing
    "device" : "Google Pixel 3",
    "os_version" : "9.0",
        
    #..
}
desired_caps = {
  
    # Specify device and os_version for testing
    "device" : "iPhone 11 Pro",
    "os_version" : "13",
        
    #..
}

You can also pass regular expressions (regex) in device and os_version capabilities if your device selection is more flexible. Check out how to use regular expressions to specify device attributes.

Organize tests

Naming your tests and builds properly is crucial for effective debugging, test reporting, and analyzing your build execution time. Following are the capabilities you can use to organize your test runs.

Capability Description
name Name of your test case. For example, Login Screen - Verify Login Button.
build CI/CD job or build name. For example, Android build #23, ios_staging_1.3.27.
project Name of your project. For example, BrowserStack Android App, iOS Consumer App.
browserstack_sample.py

We recommend that you use a new build name every time you run your test cases. Using a new build name upon every test run makes it easier to find and debug tests on the App Automate dashboard. Using a static build name across multiple test runs will group those tests into a single build on the dashboard.

A build can only have a maximum of 1000 tests and when the limit is reached, a new build gets created with a '-1' suffixed to the original build name, for example, alpha_0.1.7-1.

Mark test as passed or failed

Use JavascriptExecutor in your test script or use REST API to mark your test status as passed or failed on BrowserStack.

  • Using Javascript custom executor: Your test script might contain many assertions and you may choose to mark the status of the test based on any or all assertions. Based on your test logic, use a Javascript custom executor in your test case to mark the test status as pass or fail.

  • Using REST API: You can also mark a test session’s status as passed or failed using our REST API. This method is particularly useful when a test successfully completes on BrowserStack with a Completed status, but a failed assertion is seen in the test case.

Use the sample cURL request and update it with your session ID to update the status and reason for that session ID.

driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'

Set up debugging capabilities

BrowserStack provides a set of debugging capabilities for your tests, which are enabled by default. You can also enable following optional capabilities to further debug your tests:

  • Visual Logs: Use the visual logs capability to capture screenshots at every Appium command automatically. Enable visual logs using the browserstack.debug capability as shown in the adjacent snippet.
  • Network Logs: Network Logs capture the browser’s performance data, such as network traffic, latency, HTTP requests, and responses in a HAR format. You can enable network logs using the browserstack.networkLogs capability as shown in the following code snippet.
browserstack_sample.py

BrowserStack App Automate supports multiple other debugging capabilities including video recording, device logs, crash logs and app profiling. Check out documentation to explore all available debug options.

Commonly used features and advanced-use cases

Here’s a list of features and capabilities you may find useful for your tests.

Inspect elements with Appium Desktop

You can inspect your app elements and interact with them on BrowserStack’s real devices using Appium Desktop. BrowserStack’s integration with Appium Desktop allows you to try the inspector on real Android and iOS devices. Download the latest release of Appium Desktop from the Appium Desktop Github page and checkout detailed documentation for Appium Desktop integration.

Change device orientation

You can change the screen orientation of an Android or iOS device to set your application in portrait or landscape mode during the execution of the test. Use the browserstack.gpsLocation capability to change the screen orientation.

Capability Description Expected values
deviceOrientation Set the screen orientation of mobile device. portrait, landscape
Default: portrait    
browserstack_sample.py

Handle permission pop-ups

While testing different scenarios, it’s common for iOS or Android apps to show various pop-ups or system dialogs that prompt the user to grant various permissions, such as contacts, notifications, photos, etc., to the app.
Following are the ways to handle these pop-ups during automated test execution for Android and iOS apps:

  • Handle pop-ups for Android apps: Use Appium’s autoGrantPermissions capability to automatically determine which permissions your app requires and grant them to the app on install. These permissions are determined based on the Android manifest defined in the the app’s .apk file. This cabability works on Appium 1.9.1 and above.

  • Handle pop-ups for iOS apps: Use Appium’s autoAcceptAlerts and autoDismissAlerts capabilities to handle app permissions. autoAcceptAlerts will automatically accept all permission pop-ups and autoDismissAlerts will automatically dismiss all permission pop-ups. This includes privacy access permission pop-ups, such as location, contacts, and photos.

desired_cap = {
  "autoGrantPermissions": "true",
}
desired_cap = {
  "autoAcceptAlerts": "true", #to accept all alerts
}
#OR
desired_cap = {
  "autoDismissAlerts": "true" #to dismiss all alerts
}

Simulate GPS location

GPS Geolocation testing enables you to simulate the location of the device to specific GPS coordinates to test location-based scenarios for your app.

Capability Description Value
browserstack.gpsLocation Simulate the location of the device to a specific GPS location. Latitude followed by longitude

Acceptable range for latitude is -90 to +90 and for longitude is -180 to +180.

For example, “40.730610,-73.935242”
browserstack_sample.py

Check out all BrowserStack custom capabilities

Use our capabilities builder to configure your Appium test suite.

The capability builder lets you select from a comprehensive set of options that you can use to customize your tests on BrowserStack App Automate.

Test on different device and OS combinations

Ship high quality applications with a great user experience by testing on different real mobile devices on BrowserStack. Here is how you can choose the a list of devices to test on:

Know your target audience

Identify which devices are popular among your target audience. Select the devices according to their popularity in your targeted geographical markets. Your list should include all the important devices relevant to your market.

Consider the market share of each device before shortlisting it. For example, in the US market, the top 20 mobile devices totaling 73.43% of the total devices, consist of only iPhones and Samsung Galaxy phones.

OS & OS Version Adoption

To ensure that your app supports various platforms, test your app on different OS versions. You can refer to the analytics tool used by your organization and create a plan for the most active devices and OS versions that your customers are using.

As Apple users tend to upgrade regularly, testing on the latest iPhone or iPad model running the latest version of iOS is necessary. Android users are not as quick as iOS users to adopt the latest version of Android, and adoption is largely dictated by their carrier/device. You should definitely test devices that run one of the previous two releases of Android.

Check out the device-OS combination page to see the list of devices and OS versions currently supported.

Manufacturers

Top brands for Android devices include Samsung, Google, Xiaomi, Motorola, and One Plus. You can create an app testing device matrix to identify the combination of devices that can get you the broadest coverage in terms of OS, model, and brand. You can prioritize the devices as most important, secondary, and additional based on their relevance to your target demographics.

Other parameters:

In addition to above parameters, you also need to check for other important parameters such as:

  1. Regular Culprits: Test on the devices where using the app often leads to problems. You can prioritize these devices from your crash reports, customer feedback, and bugs data
  2. Hardware: Check how your app is affected by the processing power of hardware types of various device’s.
  3. Screen size and resolution: As the mobile devices comes in different screen sizes and resolutions, it becomes mandatory to consider the screen size, resolution, and PPI of the devices to verify UI compatibility.

Build a device coverage matrix

Compare and compile the list of devices based on the factors mentioned above. Then, prepare a list of devices with all the necessary information.

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback!

Talk to an Expert
Download Copy