Specflow
Documentation for running Selenium Tests with BrowserStack.
Getting Started
BrowserStack supports Selenium automated tests using Specflow, and running your tests on our cloud setup is simple and straightforward. Get started with a sample test, which opens Google's homepage, searches for ‘BrowserStack’, and asserts for the title of the search results page. For the code to run successfully on your machine, please ensure that the following libraries have been installed:
Install SpecFlow using NuGet Gallery
Here is a sample test case written for running with Specflow.
// Google Feature Feature: Google Scenario Outline: Can find search results Given I am on the google page for <profile> and <environment> When I search for "BrowserStack" Then I should see title "BrowserStack - Google Search" Examples: | profile | environment | | single | chrome |
// Google Steps [Binding] public class SingleSteps { private IWebDriver _driver; readonly BrowserStackDriver _bsDriver; public SingleSteps() { _bsDriver = (BrowserStackDriver)ScenarioContext.Current["bsDriver"]; } [Given(@"I am on the google page for (.*) and (.*)")] public void GivenIAmOnTheGooglePage(string profile, string environment) { _driver = _bsDriver.Init(profile, environment); _driver.Navigate().GoToUrl("https://www.google.com/ncr"); } [When(@"I search for ""(.*)""")] public void WhenISearchFor(string keyword) { var q = _driver.FindElement(By.Name("q")); q.SendKeys(keyword); q.Submit(); } [Then(@"I should see title ""(.*)""")] public void ThenIShouldSeeTitle(string title) { Thread.Sleep(5000); Assert.That(_driver.Title, Is.EqualTo(title)); } }
To actually run the test case, we need to integrate with BrowserStack as follows.
Integration with BrowserStack
To obtain your username and access keys, sign up for a Free Trial or purchase a plan.
Integration of Specflow with BrowserStack is made possible by use of following module:
[Binding] public sealed class BrowserStack { private BrowserStackDriver bsDriver; private string[] tags; [BeforeScenario] public void BeforeScenario() { bsDriver = new BrowserStackDriver(ScenarioContext.Current); ScenarioContext.Current["bsDriver"] = bsDriver; } [AfterScenario] public void AfterScenario() { bsDriver.Cleanup(); } }
public class BrowserStackDriver { private IWebDriver driver; private Local browserStackLocal; private string profile; private string environment; private ScenarioContext context; public BrowserStackDriver(ScenarioContext context) { this.context = context; } public IWebDriver Init(string profile, string environment) { NameValueCollection caps = ConfigurationManager.GetSection("capabilities/" + profile) as NameValueCollection; NameValueCollection settings = ConfigurationManager.GetSection("environments/" + environment) as NameValueCollection; DesiredCapabilities capability = new DesiredCapabilities(); foreach (string key in caps.AllKeys) { capability.SetCapability(key, caps[key]); } foreach (string key in settings.AllKeys) { capability.SetCapability(key, settings[key]); } String username = Environment.GetEnvironmentVariable("BROWSERSTACK_USERNAME"); if (username == null) { username = ConfigurationManager.AppSettings.Get("user"); } String accesskey = Environment.GetEnvironmentVariable("BROWSERSTACK_ACCESS_KEY"); if (accesskey == null) { accesskey = ConfigurationManager.AppSettings.Get("key"); } capability.SetCapability("browserstack.user", username); capability.SetCapability("browserstack.key", accesskey); File.AppendAllText("C:\\Users\\Admin\\Desktop\\sf.log", "Starting local"); if (capability.GetCapability("browserstack.local") != null && capability.GetCapability("browserstack.local").ToString() == "true") { browserStackLocal = new Local(); List<KeyValuePair<string, string>> bsLocalArgs = new List<KeyValuePair<string, string>>() { new KeyValuePair<string, string>("key", accesskey) }; browserStackLocal.start(bsLocalArgs); } File.AppendAllText("C:\\Users\\Admin\\Desktop\\sf.log", "Starting driver"); driver = new RemoteWebDriver(new Uri("http://" + ConfigurationManager.AppSettings.Get("server") + "/wd/hub/"), capability); return driver; } public void Cleanup() { driver.Quit(); if (browserStackLocal != null) { browserStackLocal.stop(); } } }
The module reads from config file where you need to put the BrowserStack Hub URL and credentials.
<?xml version="1.0" encoding="utf-8"?> <configuration> <configSections> <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" /> <sectionGroup name="capabilities"> <section name="single" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </sectionGroup> <sectionGroup name="environments"> <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </sectionGroup> </configSections> <appSettings> <add key="user" value="USERNAME" /> <add key="key" value="ACCESS_KEY" /> <add key="server" value="hub-cloud.browserstack.com" /> </appSettings> <capabilities> <single> <add key="browserstack.debug" value="true" /> </single> </capabilities> <environments> <chrome> <add key="browser" value="chrome" /> </chrome> </environments> <startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow> <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --> <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>
Run your test on BrowserStack using following command:
Build the solution in Visual Studio 2015 Update 2 Run test with fixture "single" from Test Explorer
Testing on Internal Networks
To test a private server with Specflow on BrowserStack, install local bindings.
Install BrowserstackLocal using NuGet Gallery
Then update config file and set the browserstack.local capability to true.
<?xml version="1.0" encoding="utf-8"?> <configuration> <configSections> <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" /> <sectionGroup name="capabilities"> <section name="local" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </sectionGroup> <sectionGroup name="environments"> <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </sectionGroup> </configSections> <appSettings> <add key="user" value="BROWSERSTACK_USERNAME" /> <add key="key" value="BROWSERSTACK_ACCESS_KEY" /> <add key="server" value="hub-cloud.browserstack.com" /> </appSettings> <capabilities> <local> <add key="browserstack.local" value="true" /> </local> </capabilities> <environments> <chrome> <add key="browser" value="chrome" /> </chrome> </environments> <startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow> <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --> <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>
Here is a sample test case written for running local with Specflow.
// Local Feature Feature: Local Scenario Outline: Can check tunnel working Given I opened health check for <profile> and <environment> Then I should see "Up and running" Examples: | profile | environment | | local | chrome |
// Local Steps [Binding] public class LocalSteps { private IWebDriver _driver; readonly BrowserStackDriver _bsDriver; public LocalSteps() { _bsDriver = (BrowserStackDriver)ScenarioContext.Current["bsDriver"]; } [Given(@"I opened health check for (.*) and (.*)")] public void GivenIOpenedHealthCheck(string profile, string environment) { _driver = _bsDriver.Init(profile, environment); _driver.Navigate().GoToUrl("http://bs-local.com:45691/check"); } [Then(@"I should see ""(.*)""")] public void ThenIShouldSee(string body) { Assert.That(_driver.PageSource, Does.Contain(body)); } }
Run your local test on BrowserStack using following command:
Build the solution in Visual Studio 2015 Update 2 Run test with fixture "local" from Test Explorer
Speed up testing
To run tests on multiple browsers in parallel with Specflow on BrowserStack, modify the config file as below:
// App.config <?xml version="1.0" encoding="utf-8"?> <configuration> <configSections> <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" /> <sectionGroup name="capabilities"> <section name="parallel" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </sectionGroup> <sectionGroup name="environments"> <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> <section name="firefox" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> <section name="safari" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> <section name="ie" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </sectionGroup> </configSections> <appSettings> <add key="user" value="USERNAME" /> <add key="key" value="ACCESS_KEY" /> <add key="server" value="hub-cloud.browserstack.com" /> </appSettings> <capabilities> <parallel> <add key="browserstack.debug" value="true" /> </parallel> </capabilities> <environments> <chrome> <add key="browser" value="chrome" /> </chrome> <firefox> <add key="browser" value="firefox" /> </firefox> <safari> <add key="browser" value="safari" /> </safari> <ie> <add key="browser" value="ie" /> </ie> </environments> <startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow> <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --> <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>
// Add to AssemblyInfo.cs [assembly: Parallelizable(ParallelScope.Fixtures)]
Capabilities for each environment can be customised as explained earlier.
Parallel test cases need to be written for running in parallel with Specflow.
// Google Feature for parallel run Feature: Google Scenario Outline: Can find search results Given I am on the google page for <profile> and <environment> When I search for "BrowserStack" Then I should see title "BrowserStack - Google Search" Examples: | profile | environment | | parallel | chrome | | parallel | firefox | | parallel | safari | | parallel | ie |
Run your tests in parallel on BrowserStack using following command:
Build the solution in Visual Studio 2015 Update 2 Run tests with fixture "parallel" from Test Explorer
Configuring capabilities
To run your tests on BrowserStack Automate, the tests have to be run on remote machines. Therefore, the capabilities of the WebDriver have to be changed accordingly.
Run tests on desktop and mobile browsers
Using the drop-down menus, select a combination of operating system, browser, and screen resolution. To see the order of precedence for the capabilities, please read about parameter override rules here.
You can additionally run your Selenium test scripts on real Android and iOS devices in our datacenters.
Look for the icon to select a real device.
<environments> <bs> <add key="os" value="Windows" /> <add key="os_version" value="7" /> <add key="browser" value="IE" /> <add key="browser_version" value="8.0" /> <add key="resolution" value="1024x768" /> </bs> </environments>
For a list of all supported devices, visit the Browsers and Platforms page.
Builds and projects
Keep track of all your automated tests using the build and project capabilities. Group your tests into builds, and builds further into projects.
<capabilities> <single> <add key="build" value="version1" /> <add key="project" value="newintropage" /> </single> </capabilities>
Self-signed certificates
To avoid invalid certificate errors while testing on BrowserStack Automate, set the acceptSslCerts capability in your test to true.
<capabilities> <single> <add key="acceptSslCerts" value="true" /> </single> </capabilities>
Enable and Disable Pop-ups
IETo enable the popups in IE, use the browserstack.ie.enablePopups capability.
<capabilities> <single> <add key="browserstack.ie.enablePopups" value="true" /> </single> </capabilities>Safari
To enable the popups in Safari, use the browserstack.safari.enablePopups capability.
<capabilities> <single> <add key="browserstack.safari.enablePopups" value="true" /> </single> </capabilities>
Debugging
LogsTo debug failed tests, BrowserStack provides you with raw logs, which are the console logs from the browser tests; visual logs, which capture successive screenshots of the test; and text logs to display all the steps that were performed by the test.
Live screencastWith live screencast, view ongoing tests to debug the functionality of features that are being tested. The generated videos can be recorded and downloaded for later viewing.
Logs and the live screencast help you to compare and detect any changes that may have occurred since a similar test was last run. Debugging is set to false by default. To enable logs, set the BrowserStack custom capability browserstack.debug to true.
<capabilities> <single> <add key="browserstack.debug" value="true" /> </single> </capabilities>Video recording
Videos are recorded for every BrowserStack Automate test. This feature helps you to verify and debug failed tests, confirm the functionality of tested features, and download videos for later viewing.
<capabilities> <single> <add key="browserstack.video" value="false" /> </single> </capabilities>
Other capabilities
BrowserStack supports the full complement of Selenium capabilities, as well as, some custom ones.
Additional Notes
REST API
It is possible to mark tests as either a pass or a fail, using the following snippet:
namespace RestApi { class ChangeSessionStatus { static void Main(string[] args) { string reqString = "{\"status\":\"completed\", \"reason\":\"\"}"; byte[] requestData = Encoding.UTF8.GetBytes(reqString); Uri myUri = new Uri(string.Format("https://www.browserstack.com/automate/sessions/<session-id>.json")); WebRequest myWebRequest = HttpWebRequest.Create(myUri); HttpWebRequest myHttpWebRequest = (HttpWebRequest)myWebRequest; myWebRequest.ContentType = "application/json"; myWebRequest.Method = "PUT"; myWebRequest.ContentLength = requestData.Length; using (Stream st = myWebRequest.GetRequestStream())st.Write(requestData, 0, requestData.Length); NetworkCredential myNetworkCredential = new NetworkCredential("USERNAME", "ACCESS_KEY"); CredentialCache myCredentialCache = new CredentialCache(); myCredentialCache.Add(myUri, "Basic", myNetworkCredential); myHttpWebRequest.PreAuthenticate = true; myHttpWebRequest.Credentials = myCredentialCache; myWebRequest.GetResponse().Close(); } } }
The two potential values for status can either be completed or error. Optionally, a reason can also be passed.
Queuing
With queuing, you can launch an additional number of parallel tests with different browser configurations that will be queued in a sequence, for a seamless parallel execution. For instance, if you want to run 5 additional tests, apart from your subscribed limit of 2 parallel tests, BrowserStack will queue the additional 5 tests until one of the 2 initial tests finish, and a slot is available for execution. With queuing, you can be less bothered about managing your tests, and it can save development time.
With this feature, accounts up to 5 parallel tests can queue 5 tests. Beyond 5 parallel tests, an equivalent number of tests will be queued.
We have provided examples of parallel testing implementation using popular testing frameworks. In order to increase the number of tests, purchase more parallel tests of your Automate or Automate Pro plan to get access to more tests.