Integrate your NUnit tests with BrowserStack App Automate
This section will help you setup and integrate your test suite to run on BrowserStack App Automate. It covers the key features and best practice recommendations for seamless integration.
By the end of this guide, you will have successfully learnt to:
- Setup remote Appium WebDriver
- Upload and manage apps
- Test on development or staging environments
- Test on different device and OS combinations
- Migrate your test cases
Setup remote Appium WebDriver
Set environment variables for BrowserStack credentials
BrowserStack access key credentials are required to initialise a remote Appium WebDriver. Run the command on the right in your terminal/command-prompt to store your BrowserStack access key credentials as environment variables. We recommend to use environment variables for access key management for a production-grade integration. Once you set your BrowserStack credentials through environment variables, they can be fetched from any file while running your test scripts as shown in the next step.
# Set these values in your ~/.zprofile (zsh) or ~/.profile (bash)
export BROWSERSTACK_USERNAME="YOUR_USERNAME"
export BROWSERSTACK_ACCESS_KEY="YOUR_ACCESS_KEY"
# setx.exe does not set the environment variable in the current command prompt, but it will be available in subsequent command prompts
setx BROWSERSTACK_USERNAME "YOUR_USERNAME"
setx BROWSERSTACK_ACCESS_KEY "YOUR_ACCESS_KEY"
# Verify whether the variables have been set
echo BROWSERSTACK_USERNAME
echo BROWSERSTACK_ACCESS_KEY
Update your test configuration
Next, set the value of user
and key
variables to set your BrowserStack access credentials using environment variables in your test configuration file. In addition, update the server
variable to a remote BrowserStack hub URL as shown in the code snippet on the right.
All code snippets in this guide use an example XML file (App.config
) to store key test configuration data and Appium desired capabilities. If you don’t have a config file already setup, you can download it as a base config.
Initialize remote Appium Webdriver
You are now ready to initialize an instance of Appium webdriver that can be used to run remote test sessions on BrowserStack’s real device cloud. Create a new instance of Appium webdriver using BrowserStack access credentials and the remote BrowserStack hub URL as shown in the code snippet on the right.
namespace android.first
{
public class BrowserStackNUnitTest
{
//...
protected AndroidDriver<AndroidElement> driver;
[SetUp]
public void Init()
{
//...
Uri uri = new Uri("https://" + ConfigurationManager.AppSettings.Get("server") + "/wd/hub/");
driver = new AndroidDriver<AndroidElement>(uri, options);
}
//...
}
}
namespace ios.first
{
public class BrowserStackNUnitTest
{
//...
protected IOSDriver<IOSElement> driver;
[SetUp]
public void Init()
{
//...
Uri uri = new Uri("https://" + ConfigurationManager.AppSettings.Get("server") + "/wd/hub/");
driver = new IOSDriver<IOSElement>(uri, options);
}
//...
}
}
App uploads and management
In order to test on BrowserStack App Automate, upload a test build of your app so that it can be installed on BrowserStack devices before the test execution begins.
Upload Apps on BrowserStack
It’s easy to upload an Android app (.apk
or .aab
file) or an iOS app (.ipa
file) from your local filesystem or a CI/CD server to BrowserStack using our REST API.
You can either upload an app from your file system or provide a public url using our upload app REST API endpoint.
Run the cURL
request on the right in your terminal/command-prompt to upload the app and generate a unique app ID. Take note of app_url
value, bs://{app_id}
format, in the API response. This value will be used to specify the app
capability that sets the application under test in your Appium test scripts in steps ahead.
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/upload" \
-F "file=@/path/to/app/file/application-debug.apk"
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/upload" \
-F "file=@/path/to/app/file/application-debug.ipa"
You can also refer to Custom ID feature for ease of integration. Using Custom ID removes the overhead of updating the app
capability everytime a new build of the app is uploaded.
Install the cURL utility for Windows to run the cURL commands on Windows.
Ensure that the @
symbol is prepended to the file path in the app upload API request. For example, file=@/path/to/apk/file
.
Specify Application under Test
After the app is uploaded, update the app
capability in your test configuration file with the app_url
value obtained in the above step. The uploaded app will be installed on the device during test execution.
Capability | Expected values |
---|---|
app |
Your application under test. Set to app_url value, bs://{app_id} in the API response. |
Additional information
Here’s some information that you may find useful for integration with BrowserStack.
Prepare an Android app for testing on BrowserStack
- For Android apps, the supported file formats are
.apk
and.aab
. - If you upload an unsigned version of an Android app, we sign them with our certificates before installing them to our devices. Similarly, whenever
.aab
files are uploaded, we convert them to a universalAPK
and sign them with our certificates. - When BrowserStack re-signs the apps, your app’s integration with third-party libraries, such as Google Firebase services, Google Maps SDK, Facebook SDK, etc may not work if you have restricted the use of API keys based on the SHA-1 certificate fingerprint of the app’s signing key. To avoid this issue, you can sign the
APK
with your own certificates before uploading the app to BrowserStack.
Prepare an iOS app for testing on BrowserStack
- For iOS apps, the supported file format is
.ipa
. Ensure that your app is built for running on real devices and not simulators. - When you upload an iOS app, we will re-sign the app with our own provisioning profile to be able to install your app on our devices during test execution. Because we re-sign the application before running a session with our certificate, app entitlements are removed and if the apps are relying on these entitlements, the app might crash or app functionality will stop working.
However, if your app is signed using the Apple Developer Enterprise Program, you can disable this behavior to test features, such as push notifications on BrowserStack devices. To disable re-signing, set the browserstack.resignApp
capability to false
in your Appium test scripts.
Capability | Description | Value |
---|---|---|
browserstack.resignApp |
Disable re-signing of Enterprise signed app uploaded on BrowserStack | false |
Manage uploaded apps on BrowserStack
You can manage your uploaded apps using our REST API. Use the recent apps API endpoint to list your recently uploaded apps. You can delete your uploaded apps using the delete app API endpoint. By default, we will delete the uploaded app after 30 days from the date of upload.
Test on development or staging environments
Local Testing is a BrowserStack feature that helps you test mobile apps that access resources hosted in development or testing environments during automated test execution.
You can setup localhost testing using the following two ways:
- Via adding dependency (language bindings): The BrowserStackLocal binary is installed by adding it as a dependency in the pom.xml file. It internally establishes a secure tunnel with BrowserStack when you run your test case. The package provides hooks that let you control the lifecycle of the tunnel. Using language bindings is the recommended way to integrate localhost testing on BrowserStack.
- Via CLI Interface (binary): You can also download the BrowserStack Local binary and setup the tunnel via your terminal. This method is useful when you don’t want to make any code changes in your test suite and want to manage the tunnel lifecycle independently.
- Language Bindings
- CLI Interface - Binary
Install the package
Download and add the BrowserStackLocal NuGet package.
Set up BrowserStack user name and access key
Update your NUnit configuration (.config
) file to set your BrowserStack user
and key
.
<configuration>
...
<appSettings>
<add key="user" value="<browserstack-user-name>"/>
<add key="key" value="<browserstack-access-key>"/>
</appSettings>
...
</configuration>
<configuration>
...
<appSettings>
<add key="user" value="<browserstack-user-name>"/>
<add key="key" value="<browserstack-access-key>"/>
</appSettings>
...
</configuration>
Set the uploaded app ID
Update the app
variable in your NUnit configuration (.config
) file with the app_id
of your uploaded app.
Set the access key and use available methods in your test script
Set your access key that is used to create the secure tunnel. Use the following methods provided by the local library to manage your local connection:
Method | Description |
---|---|
browserStackLocal.start(bsLocalArgs) |
Expects bsLocalArgs object. Returns a callback when the tunnel has started successfully. Your test script should start executing after this callback has been invoked. |
browserStackLocal.stop() |
Call this method after your test suite is complete. |
Add desired capabilities to enable browserstack.local
Update your NUnit configuration (.config
) file to set the browserstack.local
capability to true
. You may face errors running your test script if any other capability is enabled before setting up browserstack.local
.
If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Change for App Automate - Check out Local Binary parameters to learn about additional arguments.
Execute build on BrowserStack
You are now ready to execute your build on BrowserStack.
Download BrowserStack Local
Unzip the binary
Unzip the downloaded file and move it to a folder/directory from which you have permission to start it using your command line or terminal.
Run the binary using your command line or terminal
Run the following command to initiate the BrowserStack Local connection
# Step 3 - Run this command in your terminal to start the BrowserStack Local binary. Your working directory should be where you have the downloaded binary.
./BrowserStackLocal --key YOUR_ACCESS_KEY
# Step 3 - Run this command in your command prompt. Your working directory should be where you have unzipped BrowserStackLocal.exe
BrowserStackLocal.exe --key YOUR_ACCESS_KEY
If your staging environment is behind a proxy or firewall, additional arguments, such as proxy username, proxy password, etc, need to be set. Check out Local Binary parameters to learn about additional arguments.
Set up config to enable browserstack.local
Copy the capabilities from the adjacent window into your NUnit configuration file.
Execute build on BrowserStack
You are now ready to execute your build on BrowserStack.
Migrate your test cases
This section will help you with all the configuration changes, commonly used features, and best practices for a smooth migration of your test cases to BrowserStack.
Configure OS and Device Combinations for your tests
BrowserStack App Automate offers an extensive coverage of real iOS and Android devices for automated testing. Check out this link to see the list of devices and OS versions currently supported.
We recommend running your build using a single Android or iOS device to begin with. This step helps isolate issues during the migration phase and aids faster debugging. Once you’ve migrated your test cases or have achieved stability with a single device, you can try running the tests on multiple devices.
In order to integrate your test-suite, edit the test configuration file, for example, App.config
, to set the device
and os_version
desired capabilities to the required devices and OS versions.
Capability | Expected values |
---|---|
device |
Set to the device you want to test on. |
os_version |
Set to the OS version of the device. |
<configuration>
...
<environments>
<pixel-3>
<add key="device" value="Google Pixel 3"/>
<add key="os_version" value="9.0"/>
</pixel-3>
</environments>
...
</configuration>
<configuration>
...
<environments>
<iphone-11-pro>
<add key="device" value="iPhone 11 Pro"/>
<add key="os_version" value="13.0"/>
</iphone-11-pro>
</environments>
...
</configuration>
You can also pass regular expressions (regex) in device
and os_version
capabilities if your device selection is more flexible. Check out how to use regular expressions to specify device attributes.
Organize tests
Naming your tests and builds properly is crucial for effective debugging, test reporting, and analyzing your build execution time. Here are the capabilities you can use to organize your test runs.
Capability | Description |
---|---|
name |
Name of your test case. For example, Login Screen - Verify Login Button. |
build |
CI/CD job or build name. For example, Android build #23, ios_staging_1.3.27. |
project |
Name of your project. For example, BrowserStack Android App, iOS Consumer App. |
<configuration>
...
<capabilities>
<add key="project" value="NUnit Android"/>
<add key="build" value="First NUnit Android Test"/>
<add key="name" value="first_test"/>
</capabilities>
...
</configuration>
<configuration>
...
<capabilities>
<add key="project" value="NUnit iOS"/>
<add key="build" value="First NUnit iOS Test"/>
<add key="name" value="first_test"/>
</capabilities>
...
</configuration>
We recommend that you use a new build
name every time you run your test cases. Using a new build name upon every test run makes it easier to find and debug tests on the App Automate dashboard. Using a static build name across multiple test runs will group those tests into a single build on the dashboard.
A build can only have a maximum of 1000 tests and when the limit is reached, a new build gets created with a '-1'
suffixed to the original build name, for example, alpha_0.1.7-1
.
Mark test as passed or failed
Use JavascriptExecutor
in your test script or use REST API to mark your test status as passed or failed on BrowserStack.
Using Javascript custom executor
Your test script might contain many assertions and you may choose to mark the status of the test based on any or all assertions. Based on your test logic, use a Javascript custom executor in your test case to mark the test status as pass or fail.
Using REST API
You can also mark a test session’s status as passed or failed using our REST API. This method is particularly useful when a test successfully completes on BrowserStack with a Completed
status, but a failed assertion is seen in the test case. Use the sample cURL request and update it with your session ID to update the status and reason for that session ID.
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"<passed/failed>\", \"reason\": \" <reason> \"}}");
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'
Set up debugging capabilities
BrowserStack provides a set of debugging capabilities for your tests, which are enabled by default. You can also enable following optional capabilities to further debug your tests:
-
Visual Logs: Use the visual logs capability to capture screenshots at every Appium command automatically. Enable visual logs using the
browserstack.debug
capability as shown on the right. -
Network Logs: Network Logs capture the browser’s performance data, such as network traffic, latency, HTTP requests, and responses in a HAR format. You can enable network logs using the
browserstack.networkLogs
capability as shown on the right.
BrowserStack App Automate supports multiple other debugging capabilities including video recording, device logs, crash logs and app profiling. Check out documentation to explore all available debug options.
Commonly used features and advanced-use cases
Here’s a list of features and capabilities you may find useful for your tests.
Inspect elements with Appium Desktop
You can inspect your app elements and interact with them on BrowserStack’s real devices using Appium Desktop. BrowserStack’s integration with Appium Desktop allows you to try the inspector on real Android and iOS devices. Download the latest release of Appium Desktop from the Appium Desktop Github page and checkout detailed documentation for Appium Desktop integration.
Change device orientation
You can change the screen orientation of an Android or iOS device to set your application in portrait or landscape mode during the execution of the test. Use the deviceOrientation
capability to change the screen orientation.
Capability | Description | Expected values |
---|---|---|
deviceOrientation |
Set the screen orientation of mobile device. | portrait, landscape |
Default: portrait |
Handle permission pop-ups
While testing different scenarios, it’s common for iOS or Android apps to show various pop-ups or system dialogs that prompt the user to grant various permissions, such as contacts, notifications, photos, etc., to the app.
Here is how you can handle these interactions during automated test execution for Android and iOS apps:
Handle pop-ups for Android apps: Use Appium’s autoGrantPermissions
capability to automatically determine which permissions your app requires and grant them to the app on install. These permissions are determined based on the Android manifest defined in the the app’s .apk
file. This cabability works on Appium 1.9.1 and above.
Handle pop-ups for iOS apps: Use Appium’s autoAcceptAlerts
and autoDismissAlerts
capabilities to handle app permissions. autoAcceptAlerts
will automatically accept all permission pop-ups and autoDismissAlerts
will automatically dismiss all permission pop-ups. This includes privacy access permission pop-ups, such as location, contacts, and photos.
<configuration>
...
<capabilities>
...
<add key="autoGrantPermissions" value="true"/>
</capabilities>
...
</configuration>
<configuration>
...
<capabilities>
...
<add key="autoAcceptAlerts" value="true"/> <!-- to accept all alerts -->
<add key="autoDismissAlerts" value="true"/> <!-- to dismiss all alerts -->
</capabilities>
...
</configuration>
Simulate GPS location
GPS Geolocation testing enables you to simulate the location of the device to specific GPS coordinates to test location-based scenarios for your app.
Capability | Description | Value |
---|---|---|
browserstack.gpsLocation |
Simulate the location of the device to a specific GPS location. | Latitude followed by longitude Acceptable range for latitude is -90 to +90 and for longitude is -180 to +180. For example, “40.730610,-73.935242” |
Check out all BrowserStack custom capabilities
Use our Capabilities builder to configure your Appium test suite. The capability builder let’s you select from a comprehensive set of options that you can use to customize your tests on BrowserStack App Automate.
Test on different device and OS combinations
Ship high quality applications with a great user experience by testing on different real mobile devices on BrowserStack. Here is how you can choose the a list of devices to test on:
Know your target audience
Identify which devices are popular among your target audience. You must select the devices according to their popularity in the geographical markets you are targeting. Your list should include all the important devices relevant to your market. Take into account the market share of each device before shortlisting it. For example, in the US market, the top 20 mobile devices totaling 73.43% of the total devices, consist of only iPhones and Samsung Galaxy phones.
OS & OS Version Adoption
To ensure that your app supports various platforms, you need to test your app on different OS versions. You can refer to the analytics tool used by your organisation and create a plan for the most active devices and OS versions that your customers are using.
As Apple users tend to upgrade regularly, testing on the latest iPhone or iPad model running the latest version of iOS is a must. Android users are not as quick as iOS users to adopt the latest version of Android, and adoption is largely dictated by their carrier/device. You should definitely test devices that run one of the previous two releases of Android.
Refer to this link to see the list of devices and OS versions currently supported.
Manufacturers
Top brands for Android devices include Samsung, Google, Xiaomi, Motorola, and One Plus. You can create an app testing device matrix to identify the combination of devices that can get you the broadest coverage in terms of OS, model, and brand. You can prioritize the devices as most important, secondary, and additional based on their relevance to your target demographics.
Other parameters:
In addition to above parameters, you also need to check for other important parameters such as:
- Regular Culprits: Test on the devices where using the app often leads to problems. You can prioritize these devices from your crash reports, customer feedback, and bugs data
- Hardware: Check how your app is affected by the processing power of hardware types of various device’s.
- Screen size and resolution: As the mobile devices comes in different screen sizes and resolutions, it becomes mandatory to consider the screen size, resolution, and PPI of the devices to verify UI compatibility.
Build a device coverage matrix
Compare and compile the list of devices based on the factors mentioned above. Then, prepare a list of devices with all the necessary information.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!