Skip to main content

Integrate your test suite with BrowserStack

This section will help you setup and integrate your test suite to run on BrowserStack App Automate. It covers the key features and best practice recommendations for seamless integration.

Select your integration method

  • BrowserStack SDK (Recommended)
    Plug-n-play solution that takes care of all the integration steps for you
  • Legacy integration
    Start and stop the BrowserStack Local tunnel within your test suite manually

Using the BrowserStack SDK is the recommended integration method for Behave. The SDK handles your integration steps automatically. Use the Manual integration only when you are using custom frameworks or want to handle advanced parallelization use-cases.


By the end of this guide, you will have successfully learnt to:

  1. Setup remote Appium WebDriver
  2. Upload and manage apps
  3. Test on development or staging environments
  4. Test on different device and OS combinations
  5. Migrate your test cases

Setup remote Appium WebDriver

Set environment variables for BrowserStack credentials

BrowserStack access key credentials are required to initialise a remote Appium WebDriver. Run the command on the right in your terminal/command-prompt to store your BrowserStack access key credentials as environment variables. We recommend to use environment variables for access key management for a production-grade integration. Once you set your BrowserStack credentials through environment variables, they can be fetched from any file while running your test scripts as shown in the next step.

# Set these values in your ~/.zprofile (zsh) or ~/.profile (bash)
export BROWSERSTACK_USERNAME="YOUR_USERNAME"
export BROWSERSTACK_ACCESS_KEY="YOUR_ACCESS_KEY"
# setx.exe does not set the environment variable in the current command prompt, but it will be available in subsequent command prompts
setx BROWSERSTACK_USERNAME "YOUR_USERNAME"
setx BROWSERSTACK_ACCESS_KEY "YOUR_ACCESS_KEY"

# Verify whether the variables have been set
echo BROWSERSTACK_USERNAME
echo BROWSERSTACK_ACCESS_KEY

Update your test configuration

Next, set the value of browserstack.user and browserstack.key variables to set your BrowserStack access credentials using environment variables in your test file. In addition, you need to update the server variable to a remote BrowserStack hub URL as shown in the adjacent code snippet.

config.json

All code snippets in this guide use an example JSON file (config.json) to store key test configuration data and Appium desired capabilities.If you don’t have a config file already setup, you can download it as a base config.

Initialize remote Appium Webdriver

You are now ready to initialize an instance of Appium webdriver that can be used to run remote test sessions on BrowserStack’s real device cloud. Create a new instance of Appium webdriver using BrowserStack access credentials and the remote BrowserStack hub URL as shown in the code snippet on the right.


# Initialize the remote Webdriver using BrowserStack remote URL
# and desired capabilities defined above
 context.browser = webdriver.Remote (
        desired_capabilities=desired_capabilities,
        command_executor="http://hub.browserstack.com/wd/hub"
 )
#...

# Initialize the remote Webdriver using BrowserStack remote URL
# and desired capabilities defined above
 context.browser = webdriver.Remote (
        desired_capabilities=desired_capabilities,
        command_executor="http://hub.browserstack.com/wd/hub"
 )
#...

App uploads and management

In order to test on BrowserStack App Automate, upload a test build of your app so that it can be installed on BrowserStack devices before the test execution begins.

Upload Apps on BrowserStack

It’s easy to upload an Android app (.apk or .aab file) or an iOS app (.ipa file) from your local filesystem or a CI/CD server to BrowserStack using our REST API. You can either upload an app from your file system or provide a public url using our upload app REST API endpoint.

Run the cURL request on the right in your terminal/command-prompt to upload the app and generate a unique app ID. Take note of app_url value, bs://{app_id} format, in the API response. This value will be used to specify the app capability that sets the application under test in your Appium test scripts in steps ahead.

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/upload" \
-F "file=@/path/to/app/file/application-debug.apk"
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/upload" \
-F "file=@/path/to/app/file/application-debug.ipa"

You can also refer to Custom ID feature for ease of integration. Using Custom ID removes the overhead of updating the app capability everytime a new build of the app is uploaded.

Copy icon

Install the cURL utility for Windows to run the cURL commands on Windows.

Ensure that the @ symbol is prepended to the file path in the app upload API request. For example, file=@/path/to/apk/file.

Specify Application in your Test

After the app is uploaded, update the app capability in your test file with the app_url value obtained in the above step. The uploaded app will be installed on the device during test execution.

Capability Expected values
app Your application under test. Set to app_url value, bs://{app_id} in the API response.
config.json

Additional information

Here’s some information that you may find useful for integration with BrowserStack.

Prepare an Android app for testing on BrowserStack

  1. For Android apps, the supported file formats are .apk and .aab.
  2. If you upload an unsigned version of an Android app, we sign them with our certificates before installing them to our devices. Similarly, whenever .aab files are uploaded, we convert them to a universal APK and sign them with our certificates.
  3. When Browserstack re-signs the apps, your app’s integration with third-party libraries, such as Google Firebase services, Google Maps SDK, Facebook SDK, etc may not work if you have restricted the use of API keys based on the SHA-1 certificate fingerprint of the app’s signing key. To avoid this issue, you can sign the APK with your own certificates before uploading the app to BrowserStack.

Prepare an iOS app for testing on BrowserStack

  1. For iOS apps, the supported file format is .ipa. Ensure that your app is built for running on real devices and not simulators.
  2. When you upload an iOS app, we will re-sign the app with our own provisioning profile to be able to install your app on our devices during test execution. Because we re-sign the application before running a session with our certificate, app entitlements are removed and if the apps are relying on these entitlements, the app might crash or app functionality will stop working.

However, if your app is signed using the Apple Developer Enterprise Program, you can disable this behavior to test features, such as push notifications on BrowserStack devices. To disable re-signing, set the browserstack.resignApp capability to false in your Appium test scripts.

config.json
Capability Description Value
browserstack.resignApp Disable re-signing of Enterprise signed app uploaded on BrowserStack false

Manage uploaded apps on BrowserStack

You can manage your uploaded apps using our REST API. Use the recent apps API endpoint to list your recently uploaded apps. You can delete your uploaded apps using the delete app API endpoint. By default, we will delete the uploaded app after 30 days from the date of upload.

Test on development or staging environments

Local Testing is a BrowserStack feature that helps you test mobile apps that access resources hosted in development or testing environments during automated test execution.

You can setup localhost testing using the following two ways:

  • Via PIP (language bindings): The BrowserStack Local binary is installed using the pip command. It internally establishes a secure tunnel with BrowserStack when you run your test case. The package provides hooks that let you control the lifecycle of the tunnel. Using language bindings is the recommended way to integrate localhost testing on Browserstack.
  • Via CLI Interface (binary): You can also download the BrowserStack Local binary and setup the tunnel via your terminal. This method is useful when you don’t want to make any code changes in your test suite and want to manage the tunnel lifecycle independently.
  • Language Bindings
  • CLI Interface - Binary

Install the package

Add browserstack-local to your project using the pip command.

Set up BrowserStack user name and access key

Update your Lettuce configuration (.json) file to set your BrowserStack user name and access key. The config.json file is located in the ../examples/run_local_test/ directory.

"capabilities": {
      "browserstack.user" : "YOUR_USERNAME",
      "browserstack.key" : "YOUR_ACCESS_KEY",
    }

"capabilities": {
      "browserstack.user" : "YOUR_USERNAME",
      "browserstack.key" : "YOUR_ACCESS_KEY",

    }

Set the uploaded app ID

Update your Lettuce configuration (.json) file to set the app-id of your uploaded app. The config.json file is located in the ../examples/run_local_test/ directory.

config.json

Set the access key and use available methods

Set your access key that is used to create the secure tunnel in the terrain.py file located in the examples/run-local-test/features directory.

Use the following methods provided by the local library to manage your local connection:

Method Description
bs_local.start(**bs_local_args) Expects **bs_local_args object. Returns a callback when the tunnel has started successfully. Your test script should start executing after this callback has been invoked.
bs_local.stop() Call this method after your test suite is complete.
terrain.py

Initialize a remote Webdriver

Initialize the remote Webdriver using BrowserStack remote URL and desired capabilities in the terrain.py file located in the examples/run-local-test/features directory.

terrain.py

Add desired capabilities to enable browserstack.local

Update your Lettuce configuration (.json) file to set the browserstack.local capability to true. You may face errors running your test script if any other capability is enabled before setting up browserstack.local.
The config.json file is located in the ../examples/run_local_test/ directory.

config.json

If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Change for App Automate - Check out Local Binary parameters to learn about additional arguments.

Execute build on BrowserStack

You are now ready to execute your build on BrowserStack. On your local machine, open the terminal/command prompt. Navigate to the examples/ directory and run the command copied from the adjacent window.

Download BrowserStack Local

Show download options

Unzip the binary

Unzip the downloaded file and move it to a folder/directory from which you have permission to start it using your command line or terminal.

Run the binary using your command line or terminal

Run the following command to initiate the BrowserStack Local connection

# Step 3 - Run this command in your terminal to start the BrowserStack Local binary. Your working directory should be where you have the downloaded binary.
./BrowserStackLocal --key YOUR_ACCESS_KEY
# Step 3 - Run this command in your command prompt. Your working directory should be where you have unzipped BrowserStackLocal.exe
BrowserStackLocal.exe --key YOUR_ACCESS_KEY

If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Check out Local Binary parameters to learn about additional arguments.

Set up config to enable browserstack.local

Copy the capabilities from the adjacent window into your configuration file,config.json located in the examples/run_local_test directory.

config.json

Execute build on BrowserStack

You are now ready to execute your build on BrowserStack. On your local machine, open the terminal/command prompt. Navigate to the examples/ directory and run the command copied from the adjacent window.

Migrate your test cases

This section will help you with all the configuration changes, commonly used features, and best practices for a smooth migration of your test cases to BrowserStack.

Configure OS and Device Combinations for your tests

BrowserStack App Automate offers an extensive coverage of real iOS and Android devices for automated testing. Check out this link to see the list of devices and OS versions currently supported.

We recommend running your build using a single Android or iOS device to begin with. This step helps isolate issues during the migration phase and aids faster debugging. Once you’ve migrated your test cases or have achieved stability with a single device, you can try running the tests on multiple devices.

In order to integrate your test-suite, edit the test configuration file, for example, config.json, to set the device and os_version desired capabilities to the required devices and OS versions.

Capability Expected values
device Set to the device you want to test on.
os_version Set to the OS version of the device.
// Set these capabilities in config file config.json
"capabilities": = {
  
    // Specify device and os_version for testing
    "device" : "Google Pixel 3",
    "os_version" : "9.0",
        
    //..
}
// Set these capabilities in config file config.json
"capabilities": {
  
    // Specify device and os_version for testing
    "device" : "iPhone 11 Pro",
    "os_version" : "13",
        
    //..
}

You can also pass regular expressions (regex) in device and os_version capabilities if your device selection is more flexible. Check out how to use regular expressions to specify device attributes.

Organize tests

Naming your tests and builds properly is crucial for effective debugging, test reporting, and analyzing your build execution time. Here are the capabilities you can use to organize your test runs.

Capability Description
name Name of your test case. For example, Login Screen - Verify Login Button.
build CI/CD job or build name. For example, Android build #23, ios_staging_1.3.27.
project Name of your project. For example, BrowserStack Android App, iOS Consumer App.
// Set these capabilities in config file config.json
"capabilities": {
  "project" : "First Python Local project", 
  "build" : "browserstack-build-1",
  "name" : "first_test"
}
// Set these capabilities in config file config.json
"capabilities": = {
  "project" : "First Python Local project", 
  "build" : "browserstack-build-1",
  "name" : "first_test"
}

We recommend that you use a new build name every time you run your test cases. Using a new build name upon every test run makes it easier to find and debug tests on the App Automate dashboard. Using a static build name across multiple test runs will group those tests into a single build on the dashboard.

A build can only have a maximum of 1000 tests and when the limit is reached, a new build gets created with a '-1' suffixed to the original build name, for example, alpha_0.1.7-1.

Mark test as passed or failed

Use JavascriptExecutor in your test script or use REST API to mark your test status as passed or failed on BrowserStack.

Using Javascript custom executor

Your test script might contain many assertions and you may choose to mark the status of the test based on any or all assertions. Based on your test logic, use a Javascript custom executor in your test case to mark the test status as pass or fail.

Using REST API

You can also mark a test session’s status as passed or failed using our REST API. This method is particularly useful when a test successfully completes on BrowserStack with a Completed status, but a failed assertion is seen in the test case. Use the sample cURL request and update it with your session ID to update the status and reason for that session ID.

driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'

Set up debugging capabilities

BrowserStack provides a set of debugging capabilities for your tests, which are enabled by default. You can also enable following optional capabilities to further debug your tests:

  1. Visual Logs: Use the visual logs capability to capture screenshots at every Appium command automatically. Enable visual logs using the browserstack.debug capability as shown on the right.
  2. Network Logs: Network Logs capture the browser’s performance data, such as network traffic, latency, HTTP requests, and responses in a HAR format. You can enable network logs using the browserstack.networkLogs capability as shown on the right.
config.json

BrowserStack App Automate supports multiple other debugging capabilities including video recording, device logs, crash logs and app profiling. Checkout documentation to explore all available debug options.

Commonly used features and advanced-use cases

Here’s a list of features and capabilities you may find useful for your tests.

Inspect elements with Appium Desktop

You can inspect your app elements and interact with them on BrowserStack’s real devices using Appium Desktop. BrowserStack’s integration with Appium Desktop allows you to try the inspector on real Android and iOS devices. Download the latest release of Appium Desktop from the Appium Desktop Github page and checkout detailed documentation for Appium Desktop integration.

Change device orientation

config.json

You can change the screen orientation of an Android or iOS device to set your application in portrait or landscape mode during the execution of the test. Use the deviceOrientation capability to change the screen orientation.

Capability Description Expected values
deviceOrientation Set the screen orientation of mobile device. portrait, landscape
Default: portrait    

Handle permission pop-ups

// Set this capability in config file config.json
"capabilities": {
  "autoGrantPermissions": "true",
}
// Set this capability in config file config.json 
"capabilities": {
  "autoAcceptAlerts": "true", //to accept all alerts
}
// OR
"capabilities": {
  "autoDismissAlerts": "true" //to dismiss all alerts
}

While testing different scenarios, it’s common for iOS or Android apps to show various pop-ups or system dialogs that prompt the user to grant various permissions, such as contacts, notifications, photos, etc., to the app.
Here is how you can handle these interactions during automated test execution for Android and iOS apps:

Handle pop-ups for Android apps: Use Appium’s autoGrantPermissions capability to automatically determine which permissions your app requires and grant them to the app on install. These permissions are determined based on the Android manifest defined in the the app’s .apk file. This cabability works on Appium 1.9.1 and above.

Handle pop-ups for iOS apps: Use Appium’s autoAcceptAlerts and autoDismissAlerts capabilities to handle app permissions. autoAcceptAlerts will automatically accept all permission pop-ups and autoDismissAlerts will automatically dismiss all permission pop-ups. This includes privacy access permission pop-ups, such as location, contacts, and photos.

Simulate GPS location

config.json

GPS Geolocation testing enables you to simulate the location of the device to specific GPS coordinates to test location-based scenarios for your app.

Capability Description Value
browserstack.gpsLocation Simulate the location of the device to a specific GPS location. Latitude followed by longitude

Acceptable range for latitude is -90 to +90 and for longitude is -180 to +180.

For example, “40.730610,-73.935242”

Check out all BrowserStack custom capabilities

Use our Capabilities builder to configure your Appium test suite. The capability builder let’s you select from a comprehensive set of options that you can use to customize your tests on BrowserStack App Automate.

Test on different device and OS combinations

Ship high quality applications with a great user experience by testing on different real mobile devices on BrowserStack. Here is how you can choose the a list of devices to test on:

Know your target audience

Identify which devices are popular among your target audience. You must select the devices according to their popularity in the geographical markets you are targeting. Your list should include all the important devices relevant to your market. Take into account the market share of each device before shortlisting it. For example, in the US market, the top 20 mobile devices totaling 73.43% of the total devices, consist of only iPhones and Samsung Galaxy phones.

OS & OS Version Adoption

To ensure that your app supports various platforms, you need to test your app on different OS versions. You can refer to the analytics tool used by your organisation and create a plan for the most active devices and OS versions that your customers are using.

As Apple users tend to upgrade regularly, testing on the latest iPhone or iPad model running the latest version of iOS is a must. Android users are not as quick as iOS users to adopt the latest version of Android, and adoption is largely dictated by their carrier/device. You should definitely test devices that run one of the previous two releases of Android.

Refer to this link to see the list of devices and OS versions currently supported.

Manufacturers

Top brands for Android devices include Samsung, Google, Xiaomi, Motorola, and One Plus. You can create an app testing device matrix to identify the combination of devices that can get you the broadest coverage in terms of OS, model, and brand. You can prioritize the devices as most important, secondary, and additional based on their relevance to your target demographics.

Other parameters:

In addition to above parameters, you also need to check for other important parameters such as:

  1. Regular Culprits: Test on the devices where using the app often leads to problems. You can prioritize these devices from your crash reports, customer feedback, and bugs data
  2. Hardware: Check how your app is affected by the processing power of hardware types of various device’s.
  3. Screen size and resolution: As the mobile devices comes in different screen sizes and resolutions, it becomes mandatory to consider the screen size, resolution, and PPI of the devices to verify UI compatibility.

Build a device coverage matrix

Compare and compile the list of devices based on the factors mentioned above. Then, prepare a list of devices with all the necessary information.

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback!

Talk to an Expert
Download Copy