BrowserStack Specflow Documentation

Documentation for writing automate test scripts in Specflow.

Getting Started

Note: Refer our sample repo on Github: specflow-browserstack

BrowserStack supports Selenium automated tests using Specflow, and running your tests on our cloud setup is simple and straightforward. Get started with a sample test, which opens Google's homepage, searches for ‘BrowserStack’, and asserts for the title of the search results page. For the code to run successfully on your machine, please ensure that the following libraries have been installed:

Install SpecFlow using NuGet Gallery

Here is a sample test case written for running with Specflow.

// Google Feature
Feature: Google

Scenario Outline: Can find search results
  Given I am on the google page for <profile> and <environment>
  When I search for "BrowserStack"
  Then I should see title "BrowserStack - Google Search"
  
  Examples:
    | profile | environment |
    | single    | chrome      |

// Google Steps
[Binding]
public class SingleSteps
{
  private IWebDriver _driver;
  readonly BrowserStackDriver _bsDriver;

  public SingleSteps()
  {
    _bsDriver = (BrowserStackDriver)ScenarioContext.Current["bsDriver"];
  }

  [Given(@"I am on the google page for (.*) and (.*)")]
  public void GivenIAmOnTheGooglePage(string profile, string environment)
  {
    _driver = _bsDriver.Init(profile, environment);
    _driver.Navigate().GoToUrl("https://www.google.com/ncr");
  }

  [When(@"I search for ""(.*)""")]
  public void WhenISearchFor(string keyword)
  {
    var q = _driver.FindElement(By.Name("q"));
    q.SendKeys(keyword);
    q.Submit();
  }

  [Then(@"I should see title ""(.*)""")]
  public void ThenIShouldSeeTitle(string title)
  {
    Thread.Sleep(5000);
    Assert.That(_driver.Title, Is.EqualTo(title));
  }
}

To actually run the test case, we need to integrate with BrowserStack as follows.

Integration with BrowserStack

Note: Running your Specflow tests on BrowserStack requires a username and an access key.

To obtain your username and access keys, sign up for a Free Trial or purchase a plan.

Integration of Specflow with BrowserStack is made possible by use of following module:

[Binding]
public sealed class BrowserStack
{
  private BrowserStackDriver bsDriver;
  private string[] tags;

  [BeforeScenario]
  public void BeforeScenario()
  {
    bsDriver = new BrowserStackDriver(ScenarioContext.Current);
    ScenarioContext.Current["bsDriver"] = bsDriver;
  }

  [AfterScenario]
  public void AfterScenario()
  {
    bsDriver.Cleanup();
  }
}

public class BrowserStackDriver
{
  private IWebDriver driver;
  private Local browserStackLocal;
  private string profile;
  private string environment;
  private ScenarioContext context;

  public BrowserStackDriver(ScenarioContext context)
  {
    this.context = context;
  }

  public IWebDriver Init(string profile, string environment)
  {
    NameValueCollection caps = ConfigurationManager.GetSection("capabilities/" + profile) as NameValueCollection;
    NameValueCollection settings = ConfigurationManager.GetSection("environments/" + environment) as NameValueCollection;

    DesiredCapabilities capability = new DesiredCapabilities();

    foreach (string key in caps.AllKeys)
    {
      capability.SetCapability(key, caps[key]);
    }

    foreach (string key in settings.AllKeys)
    {
      capability.SetCapability(key, settings[key]);
    }

    String username = Environment.GetEnvironmentVariable("BROWSERSTACK_USERNAME");
    if (username == null)
    {
      username = ConfigurationManager.AppSettings.Get("user");
    }

    String accesskey = Environment.GetEnvironmentVariable("BROWSERSTACK_ACCESS_KEY");
    if (accesskey == null)
    {
      accesskey = ConfigurationManager.AppSettings.Get("key");
    }

    capability.SetCapability("browserstack.user", username);
    capability.SetCapability("browserstack.key", accesskey);

    File.AppendAllText("C:\\Users\\Admin\\Desktop\\sf.log", "Starting local");

    if (capability.GetCapability("browserstack.local") != null && capability.GetCapability("browserstack.local").ToString() == "true")
    {
      browserStackLocal = new Local();
      List<KeyValuePair<string, string>> bsLocalArgs = new List<KeyValuePair<string, string>>() {
        new KeyValuePair<string, string>("key", accesskey)
      };
      browserStackLocal.start(bsLocalArgs);
    }

    File.AppendAllText("C:\\Users\\Admin\\Desktop\\sf.log", "Starting driver");
    driver = new RemoteWebDriver(new Uri("http://" + ConfigurationManager.AppSettings.Get("server") + "/wd/hub/"), capability);
    return driver;
  }

  public void Cleanup()
  {
    driver.Quit();
    if (browserStackLocal != null)
    {
      browserStackLocal.stop();
    }
  }
}

The module reads from config file where you need to put the BrowserStack Hub URL and credentials.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />

    <sectionGroup name="capabilities">
      <section name="single" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>

    <sectionGroup name="environments">
      <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>
  </configSections>

  <appSettings>
    <add key="user" value="USERNAME" />
    <add key="key" value="ACCESS_KEY" />
    <add key="server" value="hub-cloud.browserstack.com" />
  </appSettings>

  <capabilities>
    <single>
      <add key="browserstack.debug" value="true" />
    </single>
  </capabilities>

  <environments>
    <chrome>
      <add key="browser" value="chrome" />
    </chrome>
  </environments>
  
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>

Run your test on BrowserStack using following command:

Build the solution in Visual Studio 2015 Update 2
Run test with fixture "single" from Test Explorer

Configuring capabilities

To run your tests on BrowserStack Automate, the tests have to be run on remote machines. Therefore, the capabilities of the WebDriver have to be changed accordingly.

Setting your operating system, browser, and screen resolution

Using the drop-down menus, select a combination of operating system, browser, and screen resolution. To see the order of precedence for the capabilities, please read about parameter override rules here.

1. Select an OS

Windows 7

2. Select a browser

Windows XP

3. Select a resolution

1024 x 768

<environments>
  <bs>
    <add key="os" value="Windows" />
    <add key="os_version" value="7" />
    <add key="browser" value="IE" />
    <add key="browser_version" value="8.0" />
    <add key="resolution" value="1024x768" />
  </bs>
</environments>

Note: If browser_version capability is not set, the test will run on the latest version of the browser set by browser capability.

Run tests on mobile browsers

You can run your Selenium test scripts on iOS and Android emulators by specifying the version and device in the input capabilities. These capabilities are browserName and device. The following example has iPhone selected as the browserName, and iPhone 5 as the device.

<environments>
  <bs>
    <add key="browserName" value="iPhone" />
    <add key="platform" value="MAC" />
    <add key="device" value="iPhone 5" />
  </bs>
</environments>

For a list of all supported devices, visit the Browsers and Platforms page.

Note: We have recently launched a real device cloud for Selenium testing in beta. Please contact support@browserstack.com for access.

Builds and projects

Keep track of all your automated tests using the build and project capabilities. Group your tests into builds, and builds further into projects.

<capabilities>
  <single>
    <add key="build" value="version1" />
    <add key="project" value="newintropage" />
  </single>
</capabilities>

Note: Allowed characters include uppercase and lowercase letters, digits, spaces, colons, periods, and underscores. Other characters, like hyphens or slashes are not allowed.

Self-signed certificates

To avoid invalid certificate errors while testing on BrowserStack Automate, set the acceptSslCerts capability in your test to true.

<capabilities>
  <single>
    <add key="acceptSslCerts" value="true" />
  </single>
</capabilities>

Enable and Disable Pop-ups

To enable the popups in IE, use the browserstack.ie.enablePopups capability.

<capabilities>
  <single>
    <add key="browserstack.ie.enablePopups" value="true" />
  </single>
</capabilities>

Safari

To enable the popups in Safari, use the browserstack.safari.enablePopups capability.

<capabilities>
  <single>
    <add key="browserstack.safari.enablePopups" value="true" />
  </single>
</capabilities>

Debugging

Logs

To debug failed tests, BrowserStack provides you with raw logs, which are the console logs from the browser tests; visual logs, which capture successive screenshots of the test; and text logs to display all the steps that were performed by the test.

Live screencast

With live screencast, view ongoing tests to debug the functionality of features that are being tested. The generated videos can be recorded and downloaded for later viewing.

Logs and the live screencast help you to compare and detect any changes that may have occurred since a similar test was last run. Debugging is set to false by default. To enable logs, set the BrowserStack custom capability browserstack.debug to true.

<capabilities>
  <single>
    <add key="browserstack.debug" value="true" />
  </single>
</capabilities>

Video recording

Videos are recorded for every BrowserStack Automate test. This feature helps you to verify and debug failed tests, confirm the functionality of tested features, and download videos for later viewing.

Note: Video recording increases test execution time slightly. You can disable this feature by setting the browserstack.video capability to false.

<capabilities>
  <single>
    <add key="browserstack.video" value="false" />
  </single>
</capabilities>

Other capabilities

BrowserStack supports the full complement of Selenium capabilities, as well as, some custom ones.

Running Local Test

To test a private server with Specflow on BrowserStack, install local bindings.

Install BrowserstackLocal using NuGet Gallery

Then update config file and set the browserstack.local capability to true.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />

    <sectionGroup name="capabilities">
      <section name="local" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>

    <sectionGroup name="environments">
      <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>
  </configSections>

  <appSettings>
    <add key="user" value="BROWSERSTACK_USERNAME" />
    <add key="key" value="BROWSERSTACK_ACCESS_KEY" />
    <add key="server" value="hub-cloud.browserstack.com" />
  </appSettings>

  <capabilities>
    <local>
      <add key="browserstack.local" value="true" />
    </local>
  </capabilities>

  <environments>
    <chrome>
      <add key="browser" value="chrome" />
    </chrome>
  </environments>
  
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>

Here is a sample test case written for running local with Specflow.

// Local Feature
Feature: Local

Scenario Outline: Can check tunnel working
  Given I opened health check for <profile> and <environment>
  Then I should see "Up and running"

  Examples:
    | profile | environment |
    | local   | chrome      |

// Local Steps
[Binding]
public class LocalSteps
{
  private IWebDriver _driver;
  readonly BrowserStackDriver _bsDriver;

  public LocalSteps()
  {
    _bsDriver = (BrowserStackDriver)ScenarioContext.Current["bsDriver"];
  }

  [Given(@"I opened health check for (.*) and (.*)")]
  public void GivenIOpenedHealthCheck(string profile, string environment)
  {
    _driver = _bsDriver.Init(profile, environment);
    _driver.Navigate().GoToUrl("http://bs-local.com:45691/check");
  }
      
  [Then(@"I should see ""(.*)""")]
  public void ThenIShouldSee(string body)
  {
    Assert.That(_driver.PageSource, Does.Contain(body));
  }
}

Run your local test on BrowserStack using following command:

Build the solution in Visual Studio 2015 Update 2
Run test with fixture "local" from Test Explorer

Running Parallel Tests

To run tests on multiple browsers in parallel with Specflow on BrowserStack, modify the config file as below:

// App.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />

    <sectionGroup name="capabilities">
      <section name="parallel" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>

    <sectionGroup name="environments">
      <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <section name="firefox" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <section name="safari" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <section name="ie" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>
  </configSections>

  <appSettings>
    <add key="user" value="USERNAME" />
    <add key="key" value="ACCESS_KEY" />
    <add key="server" value="hub-cloud.browserstack.com" />
  </appSettings>

  <capabilities>
    <parallel>
      <add key="browserstack.debug" value="true" />
    </parallel>
  </capabilities>

  <environments>
    <chrome>
      <add key="browser" value="chrome" />
    </chrome>
    <firefox>
      <add key="browser" value="firefox" />
    </firefox>
    <safari>
      <add key="browser" value="safari" />
    </safari>
    <ie>
      <add key="browser" value="ie" />
    </ie>
  </environments>
  
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>

// Add to AssemblyInfo.cs
[assembly: Parallelizable(ParallelScope.Fixtures)]

Capabilities for each environment can be customised as explained earlier.

Parallel test cases need to be written for running in parallel with Specflow.

// Google Feature for parallel run
Feature: Google

Scenario Outline: Can find search results
  Given I am on the google page for <profile> and <environment>
  When I search for "BrowserStack"
  Then I should see title "BrowserStack - Google Search"
  
  Examples:
    | profile | environment |
    | parallel  | chrome      |
    | parallel  | firefox     |
    | parallel  | safari      |
    | parallel  | ie          |

Run your tests in parallel on BrowserStack using following command:

Build the solution in Visual Studio 2015 Update 2
Run tests with fixture "parallel" from Test Explorer

Additional Notes

REST API

It is possible to mark tests as either a pass or a fail, using the following snippet:

namespace RestApi {
  class ChangeSessionStatus {
    static void Main(string[] args) {
      string reqString = "{\"status\":\"completed\", \"reason\":\"\"}";

      byte[] requestData = Encoding.UTF8.GetBytes(reqString);
      Uri myUri = new Uri(string.Format("https://www.browserstack.com/automate/sessions/<session-id>.json"));
      WebRequest myWebRequest = HttpWebRequest.Create(myUri);
      HttpWebRequest myHttpWebRequest = (HttpWebRequest)myWebRequest;
      myWebRequest.ContentType = "application/json";
      myWebRequest.Method = "PUT";
      myWebRequest.ContentLength = requestData.Length;
      using (Stream st = myWebRequest.GetRequestStream())st.Write(requestData, 0, requestData.Length);

      NetworkCredential myNetworkCredential = new NetworkCredential("USERNAME", "ACCESS_KEY");
      CredentialCache myCredentialCache = new CredentialCache();
      myCredentialCache.Add(myUri, "Basic", myNetworkCredential);
      myHttpWebRequest.PreAuthenticate = true;
      myHttpWebRequest.Credentials = myCredentialCache;

      myWebRequest.GetResponse().Close();
    }
  }
}

The two potential values for status can either be completed or error. Optionally, a reason can also be passed.

Queuing

With queuing, you can launch an additional number of parallel tests with different browser configurations that will be queued in a sequence, for a seamless parallel execution. For instance, if you want to run 5 additional tests, apart from your subscribed limit of 2 parallel tests, BrowserStack will queue the additional 5 tests until one of the 2 initial tests finish, and a slot is available for execution. With queuing, you can be less bothered about managing your tests, and it can save development time.

With this feature, accounts up to 5 parallel tests can queue 5 tests. Beyond 5 parallel tests, an equivalent number of tests will be queued.

Note: The wait limit for the execution of a pending queued job is 15 minutes and will be cancelled if exceeded.

We have provided examples of parallel testing implementation using popular testing frameworks. In order to increase the number of tests, purchase more parallel tests of your Automate or Automate Pro plan to get access to more tests.

Support Automate Specflow

Specflow