Integrate BrowserStack Automate with AppVeyor

A guide to help you integrate AppVeyor with the BrowserStack device cloud for running all your tests.

Introduction

AppVeyor CI is a popular continuous integration and deployment platform based on .NET applications. It makes testing, deployment, and monitoring builds very easy for developers. It is free for open-source projects.

In this guide, you’ll learn how to:

• Set up a pipeline in AppVeyor
• Integrate existing test cases
• Integrate test cases for locally hosted websites

Prerequisites

Before you can start integration, ensure that the following tasks are complete.

• An SCM repository on which you want to activate the CI pipeline.
• AppVeyor account credentials.
• You have access to the BrowserStack account credentials, namely Access Key and Username.

Set up a pipeline in AppVeyor

To create a pipeline in AppVeyor, complete the following steps:

1. Go to your AppVeyor CI Projects page and click + NEW PROJECT. The Select repository for your new project page opens.
2. On the Select repository for your new project page, select your SCM, and click + ADD to add your repository.
3. After your repository is added, click Settings, and then click Build.
4. Select trigger type for generating build, add the commands, and save the settings.
5. Click Environment, and add BrowserStack account credentials as environment variables, namely BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY.
6. In your repository, make changes as required and commit to the SCM.

The pipeline runs, and the test result is displayed on your AppVeyor Dashboard page.

Integrate existing test cases

With existing test cases, integrating BrowserStack involves editing your test cases to add BrowserStack capabilities, credentials, and, remote URL.

Though your existing test scripts include capabilities, BrowserStack also provides specific capabilities that help determine how tests are run. The following example code snippet sets session and build name, declares Access Key and Username as environment variables, and creates a remote driver connection.

Using the default APPVEYOR_BUILD_ID and APPVEYOR_BUILD_NUMBER environment variables in the BrowserStack build name helps generate a unique build name for every test run.

Set capabilities using the following code snippet:

String username = System.getenv("BROWSERSTACK_USERNAME");
String accessKey = System.getenv("BROWSERSTACK_ACCESS_KEY");
String buildName = System.getenv("APPVEYOR_BUILD_ID");
String buildNumber = System.getenv("APPVEYOR_BUILD_NUMBER");

MutableCapabilities capabilities = new MutableCapabilities();
HashMap<String, Object> browserstackOptions = new HashMap<String, Object>();
browserstackOptions.put("buildName", "BStack Build Number: " + buildNumber);
browserstackOptions.put("sessionName", "BStack Build Name: " + buildName);
browserstackOptions.put("seleniumVersion", "4.0.0");
capabilities.setCapability("bstack:options", browserstackOptions);

WebDriver driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + "@hub.browserstack.com/wd/hub"), capabilities);

username = process.env.BROWSERSTACK_USERNAME
accessKey = process.env.BROWSERSTACK_ACCESS_KEY
buildName = process.env.APPVEYOR_BUILD_ID
buildNumber = process.env.APPVEYOR_BUILD_NUMBER


var capabilities = {
  'bstack:options' : {
    "buildName" : "BStack Build Number: " + buildNumber,
    "sessionName" : "BStack Build Name: " + buildName,
    "userName" : username,
    "accessKey" : accessKey,
    "seleniumVersion" : "4.0.0",
  },
}
var driver = new webdriver.Builder().
  usingServer("https://hub.browserstack.com/wd/hub").
  withCapabilities(capabilities).
  build();

String username = Environment.GetEnvironmentVariable("BROWSERSTACK_USERNAME");
String accessKey = Environment.GetEnvironmentVariable("BROWSERSTACK_ACCESS_KEY");
String buildName = Environment.GetEnvironmentVariable("APPVEYOR_BUILD_ID");
String buildNumber = Environment.GetEnvironmentVariable("APPVEYOR_BUILD_NUMBER");

ChromeOptions capabilities = new ChromeOptions();
Dictionary<string, object> browserstackOptions = new Dictionary<string, object>();
browserstackOptions.Add("buildName", "BStack Build Number: " + buildNumber);
browserstackOptions.Add("sessionName", "BStack Build Name: " + buildName);
browserstackOptions.Add("userName", username);
browserstackOptions.Add("accessKey", accessKey);
browserstackOptions.Add("seleniumVersion", "4.0.0");
capabilities.AddAdditionalOption("bstack:options", browserstackOptions);

driver = new RemoteWebDriver(new Uri("https://hub.browserstack.com/wd/hub/"), capabilities);

$username = getenv('BROWSERSTACK_USERNAME');
$accessKey = getenv('BROWSERSTACK_ACCESS_KEY');
$buildName = getenv('APPVEYOR_BUILD_ID');
$buildNumber = getenv('APPVEYOR_BUILD_NUMBER');

$caps = array(
	'bstack:options' => array(
    "sessionName" => "BStack Build Name: ".$buildName,
    "buildName" => "BStack Build Number: ".$buildNumber,
    "seleniumVersion" => "4.0.0",
	),
);
$web_driver = RemoteWebDriver::create(
  "https://".$username.":".$accessKey."@hub.browserstack.com/wd/hub",
  $caps
);

username = os.environ.get('BROWSERSTACK_USERNAME')
accessKey = os.environ.get('BROWSERSTACK_ACCESS_KEY')
buildName = os.environ.get('APPVEYOR_BUILD_ID')
buildNumber = os.environ.get('APPVEYOR_BUILD_NUMBER')

bstack_options = {
    "sessionName" : "BStack Build Name: " + buildName,
    "buildName" : "BStack Build Number: " + buildNumber,
    "seleniumVersion" : "4.0.0",
    "userName": username,
    "accessKey": accessKey
}
options = ChromeOptions()
options.set_capability('bstack:options', bstack_options)
driver = webdriver.Remote(
    command_executor="https://hub.browserstack.com/wd/hub",
    options=options)

username = ENV['BROWSERSTACK_USERNAME']
accessKey = ENV['BROWSERSTACK_ACCESS_KEY']
buildName = ENV['APPVEYOR_BUILD_ID']
buildNumber = ENV['APPVEYOR_BUILD_NUMBER']
  
# to run on Chrome
options = Selenium::WebDriver::Options.chrome
capabilities = {
  'bstack:options' => {
    "sessionName" => "BStack Build Name: " + buildName,
    "buildName" => "BStack Build Number: " + buildNumber,
    "seleniumVersion" => "4.0.0",
  },
}

options.add_option('bstack:options', bstack_options)
driver = Selenium::WebDriver.for(:remote,
  :url => "https://"+username+":"+accessKey+"@hub.browserstack.com/wd/hub",
  :capabilities => options)

my $username = $ENV{"BROWSERSTACK_USERNAME"};
my $accessKey = $ENV{"BROWSERSTACK_ACCESS_KEY"};
my $buildName = $ENV{"APPVEYOR_BUILD_ID"};
my $buildNumber = $ENV{"APPVEYOR_BUILD_NUMBER"};

my $caps = {
  'bstack:options' => {
    "buildName" => "BStack Build Number:".$buildNumber,
    "sessionName" => "BStack Build Number:".$buildName,
    "seleniumVersion" => "4.0.0",
    },
  }

my $host = "$username:$accessKey\@hub.browserstack.com";
my $driver = new Selenium::Remote::Driver('remote_server_addr' => $host,
  'port' => '80', 'extra_capabilities' => $caps);

String username = System.getenv("BROWSERSTACK_USERNAME");
String accessKey = System.getenv("BROWSERSTACK_ACCESS_KEY");
String buildName = System.getenv("APPVEYOR_BUILD_ID");
String buildNumber = System.getenv("APPVEYOR_BUILD_NUMBER");

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("build", "BStack Build Number: " + buildNumber);
capabilities.setCapability("name", "BStack Build Name: " + buildName);
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accessKey + "@hub.browserstack.com/wd/hub"), capabilities);

username = process.env.BROWSERSTACK_USERNAME
accessKey = process.env.BROWSERSTACK_ACCESS_KEY
buildName = process.env.APPVEYOR_BUILD_ID
buildNumber = process.env.APPVEYOR_BUILD_NUMBER

var capabilities = {
 "browserstack.user" : username,
 "browserstack.key" : accessKey,
 "build" : "BStack Build Number: " + buildNumber
 "name" : "BStack Build Name: " + buildName
}

var driver = new webdriver.Builder().
  usingServer("https://hub.browserstack.com/wd/hub").
  withCapabilities(capabilities).
  build();

String username = Environment.GetEnvironmentVariable("BROWSERSTACK_USERNAME");
String accessKey = Environment.GetEnvironmentVariable("BROWSERSTACK_ACCESS_KEY");
String buildName = Environment.GetEnvironmentVariable("APPVEYOR_BUILD_ID");
String buildNumber = Environment.GetEnvironmentVariable("APPVEYOR_BUILD_NUMBER");

OpenQA.Selenium.Chrome.ChromeOptions capability = new OpenQA.Selenium.Chrome.ChromeOptions();
capability.AddAdditionalCapability("build", "BStack Build Number: " + buildNumber, true); // CI/CD job or build name
capability.AddAdditionalCapability("name", "BStack Build Name: " + buildName, true); // test name
capability.AddAdditionalCapability("browserstack.user", username, true);
capability.AddAdditionalCapability("browserstack.key", accessKey, true);

driver = new RemoteWebDriver(
  new Uri("https://hub.browserstack.com/wd/hub/"), capability
);

$username = getenv('BROWSERSTACK_USERNAME');
$accessKey = getenv('BROWSERSTACK_ACCESS_KEY');
$buildName = getenv('APPVEYOR_BUILD_ID');
$buildNumber = getenv('APPVEYOR_BUILD_NUMBER');

$caps = array(
  "name" => "BStack Build Name: ".$buildName , // test name
  "build" => "BStack Build Number: ".$buildNumber // CI/CD job or build name
);
$web_driver = RemoteWebDriver::create(
  "https://".$username.":".$accessKey."@hub.browserstack.com/wd/hub",
  $caps
);

username = os.environ.get('BROWSERSTACK_USERNAME')
accessKey = os.environ.get('BROWSERSTACK_ACCESS_KEY')
buildName = os.environ.get('APPVEYOR_BUILD_ID')
buildNumber = os.environ.get('APPVEYOR_BUILD_NUMBER')

desired_cap = {
 'browser': 'Chrome',
 'browser_version': 'latest',
 'os': 'Windows',
 'os_version' : '10',
 'name': "BStack Build Name: " + buildName, # test name
 'build': "BStack Build Number: " + buildNumber, # CI/CD job or build name
 'browserstack.user': username,
 'browserstack.key': accessKey,
}
driver = webdriver.Remote(
    command_executor='https://hub-cloud.browserstack.com/wd/hub',
    desired_capabilities=desired_cap)

username = ENV['BROWSERSTACK_USERNAME']
accessKey = ENV['BROWSERSTACK_ACCESS_KEY']
buildName = ENV['APPVEYOR_BUILD_ID']
buildNumber = ENV['APPVEYOR_BUILD_NUMBER']

caps = Selenium::WebDriver::Remote::Capabilities.new
caps['name'] = 'BStack Build Name: ' + buildName, # test name
caps['build'] = 'BStack Build Number: ' + buildNumber # CI/CD job or build name

driver = Selenium::WebDriver.for(:remote,
  :url => "https://"+username+":"+accessKey+"@hub.browserstack.com/wd/hub",
  :desired_capabilities => caps)

my $username = $ENV{"BROWSERSTACK_USERNAME"};
my $accessKey = $ENV{"BROWSERSTACK_ACCESS_KEY"};
my $buildName = $ENV{"APPVEYOR_BUILD_ID"};
my $buildNumber = $ENV{"APPVEYOR_BUILD_NUMBER"};

my $caps = {
  "build" => "BStack Build Number:".$buildNumber,
  "name" => "BStack Build Number:".$buildName
};

my $host = "$username:$accessKey\@hub.browserstack.com";

my $driver = new Selenium::Remote::Driver('remote_server_addr' => $host,
  'port' => '80', 'extra_capabilities' => $caps);

Integrate test cases for locally hosted websites

If you are testing websites hosted locally as part of your testing or development environment, you need to configure your AppVeyor pipeline to use the Local testing.

Using the Local testing feature of BrowserStack, remote browsers at the BrowserStack cloud can access websites hosted on your private or internal networks. To learn more about how Local testing works, check out the Local testing guide.

Apart from setting up a Local connection, you must also add the browserstack.local capability in your test scripts.

This section guides how to:

1. Enable Local testing
2. Add the browserstack.local capability to test scripts

Enable Local Testing

You can enable Local testing using the ways as shown in the following tab:

    # For Linux-based systems, add the following commands in the given console to download the binary, run it, and stop its execution after the test has been executed.
    $ wget "https://www.browserstack.com/browserstack-local/BrowserStackLocal-linux-x64.zip"
    $ unzip BrowserStackLocal-linux-x64.zip
    $ ./BrowserStackLocal --key $BROWSERSTACK_ACCESS_KEY --daemon start
    $ <your-test-command>
    $ ./BrowserStackLocal --key $BROWSERSTACK_ACCESS_KEY --daemon stop

    # For macOS-based systems, add the following commands in the given console to download the binary, run it, and stop its execution after the test has been executed.
    $ wget "https://www.browserstack.com/browserstack-local/BrowserStackLocal-darwin-x64.zip"
    $ unzip BrowserStackLocal-darwin-x64.zip
    $ ./BrowserStackLocal --key $BROWSERSTACK_ACCESS_KEY --daemon start
    $ <your-test-command>
    $ ./BrowserStackLocal --key $BROWSERSTACK_ACCESS_KEY --daemon stop

    # For Windows-based systems, add the following commands in the given console to download the binary, run it, and stop its execution after the test has been executed.
    $ wget "https://www.browserstack.com/browserstack-local/BrowserStackLocal-win32.zip"
    $ powershell.exe Expand-Archive BrowserStackLocal-win32.zip
    $ .\BrowserStackLocal-win32\BrowserStackLocal.exe --key %BROWSERSTACK_ACCESS_KEY% --daemon start
    $ <your-test-command>
    $ .\BrowserStackLocal-win32\BrowserStackLocal.exe --key %BROWSERSTACK_ACCESS_KEY% --daemon stop

Add browserstack.local capability to test scripts

Add the browserstack.local capability to test scripts using the following code snippets. When you set this capability to true, BrowserStack resolves the request via the Local agent running in your network.

browserstackOptions.put("local", "true");

'bstack:options' : {
	"local" : "true",
},

browserstackOptions.Add("local", "true");

'bstack:options' => array(
	"local" => "true",
),

'bstack:options' : {
	"local" : "true",
},

'bstack:options' => {
	"local" => "true",
},

'bstack:options' : {
	"local" : "true",
},

capabilities.setCapability("browserstack.local", "true");

"browserstack.local" : "true",

capability.AddAdditionalCapability("browserstack.local", "true", true);

"browserstack.local" => "true",

"browserstack.local" : "true",

caps["browserstack.local"] = "true",

"browserstack.local" => "true",

After these configurations, when you commit any code change, AppVeyor automatically runs the test scripts as defined.

You can verify if the test passed or failed on the AppVeyor Dashboard page.