Integrate your test suite with BrowserStack
This section will help you setup and integrate your test suite to run on BrowserStack App Automate. It covers the key features and best practice recommendations for seamless integration.
Setup remote Appium WebDriver
Set environment variables for BrowserStack credentials
BrowserStack Username and Access Key are required to initialize a remote Appium WebDriver.
In the getting started section, we set up BrowserStack credentials directly in the test script. That method works for a sample build, but for a production-grade integration we recommend you store your credentials as environment variables and use those environment variables in your test script.
# Set these values in your ~/.zprofile (zsh) or ~/.profile (bash)
export BROWSERSTACK_USERNAME="YOUR_USERNAME"
export BROWSERSTACK_ACCESS_KEY="YOUR_ACCESS_KEY"
# setx.exe does not set the environment variable in the current command prompt, but it will be available in subsequent command prompts
setx BROWSERSTACK_USERNAME "YOUR_USERNAME"
setx BROWSERSTACK_ACCESS_KEY "YOUR_ACCESS_KEY"
# Verify whether the variables have been set
echo BROWSERSTACK_USERNAME
echo BROWSERSTACK_ACCESS_KEY
Update your test configuration
Use the userName
and accessKey
variables to set your BrowserStack Username and Access key via environment variables in your test script file.
desired_cap = {
"bstack:options": {
# Set your access credentials
"userName" : os.environ["BROWSERSTACK_USERNAME"],
"accessKey" : os.environ["BROWSERSTACK_ACCESS_KEY"],
}
}
import os
desired_cap = {
# Set your access credentials
"browserstack.user" : os.environ["BROWSERSTACK_USERNAME"],
"browserstack.key" : os.environ["BROWSERSTACK_ACCESS_KEY"],
#..
}
Initialize remote Appium Webdriver
You need to initialize an instance of Appium webdriver to run remote test sessions on BrowserStack’s real device cloud.
Create a new instance of Appium webdriver using BrowserStack credentials and the remote BrowserStack hub URL as shown in the following code snippet.
Upload your app
Upload app on BrowserStack
Use the following option to upload an Android app (.apk
or .aab
file) or iOS app (.ipa
file) to BrowserStack servers.
- Upload via REST API
Use our REST API to upload an Android app (.apk
or .aab
file) or an iOS app (
.ipa
file) from your local filesystem. Use the command-line tool cURL
to upload
your app as shown below :
The generated app_url
is a unique ID used to identify the uploaded app build. Take note of this app_url
as it will be used in steps ahead.
Specify application under test
After the app is uploaded, update the app
capability in your test file with the app_url
value obtained in the previous step. The uploaded app will be installed on the device during test execution.
Capability | Expected values |
---|---|
app |
Your application under test. Set to app_url value, bs://<app-id> in the API response. |
Additional information
Here’s some information that you may find useful for integration with BrowserStack.
Prepare an Android app for testing on BrowserStack
- For Android apps, the supported file formats are
.apk
and.aab
. - If you upload an unsigned version of an Android app, we sign them with our certificates before installing them to our devices. Similarly, whenever
.aab
files are uploaded, we convert them to a universalAPK
and sign them with our certificates. - When BrowserStack re-signs the apps, your app’s integration with third-party libraries, such as Google Firebase services, Google Maps SDK, Facebook SDK, etc may not work if you have restricted the use of API keys based on the SHA-1 certificate fingerprint of the app’s signing key. To avoid this issue, you can sign the
APK
with your own certificates before uploading the app to BrowserStack.
Prepare an iOS app for testing on BrowserStack
- For iOS apps, the supported file format is
.ipa
. Ensure that your app is built for running on real devices and not simulators. - When you upload an iOS app, we will re-sign the app with our own provisioning profile to be able to install your app on our devices during test execution. Because we re-sign the application before running a session with our certificate, app entitlements are removed and if the apps are relying on these entitlements, the app might crash or app functionality will stop working.
However, if your app is signed using the Apple Developer Enterprise Program, you can disable this behavior to test features, such as push notifications on BrowserStack devices. To disable re-signing, set the browserstack.resignApp
capability to false
in your Appium test scripts.
Capability | Description | Value |
---|---|---|
resignApp for W3C protocol browserstack.resignApp for JSON Wire protocol |
Disable re-signing of Enterprise signed app uploaded on BrowserStack |
true , false Default: false
|
desired_cap = {
'bstack:options' {
#...
"resignApp": "false"
#...
}
}
desired_cap = {
#...
"browserstack.resignApp": "false"
#...
}
Manage uploaded apps on BrowserStack
You can manage your uploaded apps using our REST API.
To list your recently uploaded apps, you can use the following options:
- Run the recent apps API request
- Go to the App Automate’s capability builder page and click the Application dropdown.
You can delete your uploaded apps using the delete app API endpoint. By default, we will delete the uploaded app after 30 days from the date of upload.
Test on development or staging environments
Local Testing is a BrowserStack feature that helps you test mobile apps that access resources hosted in development or testing environments during automated test execution.
You can setup localhost testing using the following two ways:
-
Via PIP (language bindings): The BrowserStack Local binary is installed using the
pip
command. It internally establishes a secure tunnel with BrowserStack when you run your test case. The package provides hooks that let you control the lifecycle of the tunnel. Using language bindings is the recommended way to integrate localhost testing on BrowserStack. - Via CLI Interface (binary): You can also download the BrowserStack Local binary and setup the tunnel via your terminal. This method is useful when you don’t want to make any code changes in your test suite and want to manage the tunnel lifecycle independently.
Install the package
Add browserstack-local
to your project using the pip
command.
Set the access key and use available methods in your test script
Set your access key that is used to create the secure tunnel. Use the following methods provided by the local library to manage your local connection:
Method | Description |
---|---|
bs_local.start(**bs_local_args) |
Expects **bs_local_args object. Returns a callback when the tunnel has started successfully. Your test script should start executing after this callback has been invoked. |
bs_local.stop() |
Call this method after your test suite is complete. |
Add desired capabilities to enable local
Update your Python test (browserstack_sample_local.py
) file to set the local
capability to true
. You may face errors running your test script if any other capability is enabled before setting up local
.
desired_cap = {
"bstack:options": {
#Set the local capability as true
"local" : "true"
}
}
desired_caps = {
#Set BrowserStack Local capability as True
"browserstack.local": "true"
}
If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Check out Local Binary parameters to learn about additional arguments.
Execute build on BrowserStack
You are now ready to execute your build on BrowserStack. On your local machine, open the terminal/command prompt. Navigate to either the android
or ios
directory and run the test script.
Download BrowserStack Local
Unzip the binary
Unzip the downloaded file and move it to a folder/directory from which you have permission to start it using your command line or terminal.
Run the binary using your command line or terminal
Run the following command to initiate the BrowserStack Local connection
If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Check out Local Binary parameters to learn about additional arguments.
# Step 3 - Run this command in your terminal to start the BrowserStack Local binary. Your working directory should be where you have the downloaded binary.
./BrowserStackLocal --key YOUR_ACCESS_KEY
# Step 3 - Run this command in your command prompt. Your working directory should be where you have unzipped BrowserStackLocal.exe
BrowserStackLocal.exe --key YOUR_ACCESS_KEY
Set up config to enable local
Copy the capabilities from the adjacent window into your test file.
desired_cap = {
"bstack:options": {
#Set the local capability as true
"local" : "true"
}
}
desired_caps = {
#Set BrowserStack Local capability as True
"browserstack.local": "true"
}
Execute build on BrowserStack
You are now ready to execute your build on BrowserStack. On your local machine, open the terminal/command prompt. Navigate to either the android
or ios
directory and run the test script.
Migrate your test cases
This section will help you with all the configuration changes, commonly used features, and best practices for a smooth migration of your test cases to BrowserStack.
Configure OS and Device Combinations for your tests
BrowserStack App Automate offers an extensive coverage of real iOS and Android devices for automated testing. Check out the list of devices and OS versions currently supported by BrowserStack.
We recommend running your build using a single Android or iOS device to begin with. This step helps isolate issues during the migration phase and aids faster debugging. Once you’ve migrated your test cases or have achieved stability with a single device, you can try running the tests on multiple devices.
To integrate your test suite, edit the test file to set the deviceName
and platformVersion
capabilities to the required devices and OS version.
Capability | Expected values |
---|---|
deviceName |
Set to the device you want to test on. |
platformVersion |
Set to the OS version of the device. |
desired_caps = {
# Specify deviceName and platformVersion for testing
"deviceName" : "Google Pixel 3",
"platformVersion" : "9.0",
#..
}
desired_caps = {
# Specify deviceName and platformVersion for testing
"deviceName" : "iPhone 11 Pro",
"platformVersion" : "13",
#..
}
You can also pass regular expressions (regex) in deviceName
and platformVersion
capabilities if your device selection is more flexible. Check out how to use regular expressions to specify device attributes.
Organize tests
Naming your tests and builds properly is crucial for effective debugging, test reporting, and analyzing your build execution time. Following are the capabilities you can use to organize your test runs.
Capability | Description |
---|---|
sessionName |
Name of your test case. For example, Login Screen - Verify Login Button. |
buildName |
CI/CD job or build name. For example, Android build #23, ios_staging_1.3.27. |
projectName |
Name of your project. For example, BrowserStack Android App, iOS Consumer App. |
We recommend that you use a new build
name every time you run your test cases. Using a new build name upon every test run makes it easier to find and debug tests on the App Automate dashboard. Using a static build name across multiple test runs will group those tests into a single build on the dashboard.
A build can only have a maximum of 1000 tests and when the limit is reached, a new build gets created with a '-1'
suffixed to the original build name, for example, alpha_0.1.7-1
.
Mark test as passed or failed
Use JavascriptExecutor
in your test script or use REST API to mark your test status as passed or failed on BrowserStack.
-
Using Javascript custom executor: Your test script might contain many assertions and you may choose to mark the status of the test based on any or all assertions. Based on your test logic, use a Javascript custom executor in your test case to mark the test status as pass or fail.
-
Using REST API: You can also mark a test session’s status as passed or failed using our REST API. This method is particularly useful when a test successfully completes on BrowserStack with a
Completed
status, but a failed assertion is seen in the test case. Use the sample cURL request and update it with your session ID to update the status and reason for that session ID.
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'
Set up debugging capabilities
BrowserStack provides a set of debugging capabilities for your tests, which are enabled by default. You can also enable following optional capabilities to further debug your tests:
-
Visual Logs: Use the visual logs capability to capture screenshots at every Appium command automatically. Enable visual logs using the
debug
capability as shown in the adjacent snippet. -
Network Logs: Network Logs capture the browser’s performance data, such as network traffic, latency, HTTP requests, and responses in a HAR format. You can enable network logs using the
networkLogs
capability as shown in the following code snippet.
BrowserStack App Automate supports multiple other debugging capabilities including video recording, device logs, crash logs and app profiling. Check out documentation to explore all available debug options.
Commonly used features and advanced use cases
Here’s a list of features and capabilities you may find useful for your tests.
Inspect elements with Appium Desktop
You can inspect your app elements and interact with them on BrowserStack’s real devices using Appium Desktop. BrowserStack’s integration with Appium Desktop allows you to try the inspector on real Android and iOS devices. Download the latest release of Appium Desktop from the Appium Desktop Github page and checkout detailed documentation for Appium Desktop integration.
Change device orientation
You can change the screen orientation of an Android or iOS device to set your application in portrait or landscape mode during the execution of the test. Use the deviceOrientation
capability to change the screen orientation.
Capability | Description | Expected values |
---|---|---|
deviceOrientation |
Set the screen orientation of mobile device. | portrait, landscape |
Default: portrait |
Handle permission pop-ups
While testing different scenarios, it’s common for iOS or Android apps to show various pop-ups or system dialogs that prompt the user to grant various permissions, such as contacts, notifications, photos, etc., to the app.
Following are the ways to handle these pop-ups during automated test execution for Android and iOS apps:
-
Handle pop-ups for Android apps: Use Appium’s
autoGrantPermissions
capability to automatically determine which permissions your app requires and grant them to the app on install. These permissions are determined based on the Android manifest defined in the the app’s.apk
file. This cabability works on Appium 1.9.1 and above. -
Handle pop-ups for iOS apps: Use Appium’s
autoAcceptAlerts
andautoDismissAlerts
capabilities to handle app permissions.autoAcceptAlerts
will automatically accept all permission pop-ups andautoDismissAlerts
will automatically dismiss all permission pop-ups. This includes privacy access permission pop-ups, such as location, contacts, and photos.
desired_cap = {
'bstack:options': {
"autoGrantPermissions": "true",
}
}
desired_cap = {
'bstack:options': {
"autoAcceptAlerts": "true", #to accept all alerts
}
#OR
'bstack:options': {
"autoDismissAlerts": "true" #to dismiss all alerts
}
}
Simulate GPS location
GPS Geolocation testing enables you to simulate the location of the device to specific GPS coordinates to test location-based scenarios for your app.
Capability | Description | Value |
---|---|---|
gpsLocation |
Simulate the location of the device to a specific GPS location. | Latitude followed by longitude Acceptable range for latitude is -90 to +90 and for longitude is -180 to +180. For example, “40.730610,-73.935242” |
Check out all BrowserStack custom capabilities
Use our capabilities builder to configure your Appium test suite.
The capability builder lets you select from a comprehensive set of options that you can use to customize your tests on BrowserStack App Automate.
Configure OS and Device Combinations for your tests
BrowserStack App Automate offers an extensive coverage of real iOS and Android devices for automated testing. Check out the list of devices and OS versions currently supported by BrowserStack.
We recommend running your build using a single Android or iOS device to begin with. This step helps isolate issues during the migration phase and aids faster debugging. Once you’ve migrated your test cases or have achieved stability with a single device, you can try running the tests on multiple devices.
To integrate your test suite, edit the test file to set the device
and os_version
capabilities to the required devices and OS version.
Capability | Expected values |
---|---|
device |
Set to the device you want to test on. |
os_version |
Set to the OS version of the device. |
desired_caps = {
# Specify device and os_version for testing
"device" : "Google Pixel 3",
"os_version" : "9.0",
#..
}
desired_caps = {
# Specify device and os_version for testing
"device" : "iPhone 11 Pro",
"os_version" : "13",
#..
}
You can also pass regular expressions (regex) in device
and os_version
capabilities if your device selection is more flexible. Check out how to use regular expressions to specify device attributes.
Organize tests
Naming your tests and builds properly is crucial for effective debugging, test reporting, and analyzing your build execution time. Following are the capabilities you can use to organize your test runs.
Capability | Description |
---|---|
name |
Name of your test case. For example, Login Screen - Verify Login Button. |
build |
CI/CD job or build name. For example, Android build #23, ios_staging_1.3.27. |
project |
Name of your project. For example, BrowserStack Android App, iOS Consumer App. |
We recommend that you use a new build
name every time you run your test cases. Using a new build name upon every test run makes it easier to find and debug tests on the App Automate dashboard. Using a static build name across multiple test runs will group those tests into a single build on the dashboard.
A build can only have a maximum of 1000 tests and when the limit is reached, a new build gets created with a '-1'
suffixed to the original build name, for example, alpha_0.1.7-1
.
Mark test as passed or failed
Use JavascriptExecutor
in your test script or use REST API to mark your test status as passed or failed on BrowserStack.
-
Using Javascript custom executor: Your test script might contain many assertions and you may choose to mark the status of the test based on any or all assertions. Based on your test logic, use a Javascript custom executor in your test case to mark the test status as pass or fail.
-
Using REST API: You can also mark a test session’s status as passed or failed using our REST API. This method is particularly useful when a test successfully completes on BrowserStack with a
Completed
status, but a failed assertion is seen in the test case.
Use the sample cURL request and update it with your session ID to update the status and reason for that session ID.
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'
Set up debugging capabilities
BrowserStack provides a set of debugging capabilities for your tests, which are enabled by default. You can also enable following optional capabilities to further debug your tests:
-
Visual Logs: Use the visual logs capability to capture screenshots at every Appium command automatically. Enable visual logs using the
browserstack.debug
capability as shown in the adjacent snippet. -
Network Logs: Network Logs capture the browser’s performance data, such as network traffic, latency, HTTP requests, and responses in a HAR format. You can enable network logs using the
browserstack.networkLogs
capability as shown in the following code snippet.
BrowserStack App Automate supports multiple other debugging capabilities including video recording, device logs, crash logs and app profiling. Check out documentation to explore all available debug options.
Commonly used features and advanced-use cases
Here’s a list of features and capabilities you may find useful for your tests.
Inspect elements with Appium Desktop
You can inspect your app elements and interact with them on BrowserStack’s real devices using Appium Desktop. BrowserStack’s integration with Appium Desktop allows you to try the inspector on real Android and iOS devices. Download the latest release of Appium Desktop from the Appium Desktop Github page and checkout detailed documentation for Appium Desktop integration.
Change device orientation
You can change the screen orientation of an Android or iOS device to set your application in portrait or landscape mode during the execution of the test. Use the browserstack.gpsLocation
capability to change the screen orientation.
Capability | Description | Expected values |
---|---|---|
deviceOrientation |
Set the screen orientation of mobile device. | portrait, landscape |
Default: portrait |
Handle permission pop-ups
While testing different scenarios, it’s common for iOS or Android apps to show various pop-ups or system dialogs that prompt the user to grant various permissions, such as contacts, notifications, photos, etc., to the app.
Following are the ways to handle these pop-ups during automated test execution for Android and iOS apps:
-
Handle pop-ups for Android apps: Use Appium’s
autoGrantPermissions
capability to automatically determine which permissions your app requires and grant them to the app on install. These permissions are determined based on the Android manifest defined in the the app’s.apk
file. This cabability works on Appium 1.9.1 and above. -
Handle pop-ups for iOS apps: Use Appium’s
autoAcceptAlerts
andautoDismissAlerts
capabilities to handle app permissions.autoAcceptAlerts
will automatically accept all permission pop-ups andautoDismissAlerts
will automatically dismiss all permission pop-ups. This includes privacy access permission pop-ups, such as location, contacts, and photos.
desired_cap = {
"autoGrantPermissions": "true",
}
desired_cap = {
"autoAcceptAlerts": "true", #to accept all alerts
}
#OR
desired_cap = {
"autoDismissAlerts": "true" #to dismiss all alerts
}
Simulate GPS location
GPS Geolocation testing enables you to simulate the location of the device to specific GPS coordinates to test location-based scenarios for your app.
Capability | Description | Value |
---|---|---|
browserstack.gpsLocation |
Simulate the location of the device to a specific GPS location. | Latitude followed by longitude Acceptable range for latitude is -90 to +90 and for longitude is -180 to +180. For example, “40.730610,-73.935242” |
Check out all BrowserStack custom capabilities
Use our capabilities builder to configure your Appium test suite.
The capability builder lets you select from a comprehensive set of options that you can use to customize your tests on BrowserStack App Automate.
Test on different device and OS combinations
Ship high quality applications with a great user experience by testing on different real mobile devices on BrowserStack. Here is how you can choose the a list of devices to test on:
Know your target audience
Identify which devices are popular among your target audience. Select the devices according to their popularity in your targeted geographical markets. Your list should include all the important devices relevant to your market.
Consider the market share of each device before shortlisting it. For example, in the US market, the top 20 mobile devices totaling 73.43% of the total devices, consist of only iPhones and Samsung Galaxy phones.
OS & OS Version Adoption
To ensure that your app supports various platforms, test your app on different OS versions. You can refer to the analytics tool used by your organization and create a plan for the most active devices and OS versions that your customers are using.
As Apple users tend to upgrade regularly, testing on the latest iPhone or iPad model running the latest version of iOS is necessary. Android users are not as quick as iOS users to adopt the latest version of Android, and adoption is largely dictated by their carrier/device. You should definitely test devices that run one of the previous two releases of Android.
Check out the device-OS combination page to see the list of devices and OS versions currently supported.
Manufacturers
Top brands for Android devices include Samsung, Google, Xiaomi, Motorola, and One Plus. You can create an app testing device matrix to identify the combination of devices that can get you the broadest coverage in terms of OS, model, and brand. You can prioritize the devices as most important, secondary, and additional based on their relevance to your target demographics.
Other parameters:
In addition to above parameters, you also need to check for other important parameters such as:
- Regular Culprits: Test on the devices where using the app often leads to problems. You can prioritize these devices from your crash reports, customer feedback, and bugs data
- Hardware: Check how your app is affected by the processing power of hardware types of various device’s.
- Screen size and resolution: As the mobile devices comes in different screen sizes and resolutions, it becomes mandatory to consider the screen size, resolution, and PPI of the devices to verify UI compatibility.
Build a device coverage matrix
Compare and compile the list of devices based on the factors mentioned above. Then, prepare a list of devices with all the necessary information.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!