Mark Tests as Passed or Failed

Mark your Automate tests as passed or failed from within the test script or after the test has completed.

BrowserStack does not know whether your test’s assertions have passed or failed because only the test script knows whether the assertions have passed. Therefore, based on the assertions on your script, you have to explicitly inform BrowserStack whether your tests have passed or not and this document will help you in doing that exactly.

Once you have executed the code snippets as shown below, you will be able to view the status of your tests on the Automate Dashboard as the example shown below:

Mark tests as passed or failed

You can mark the status of your test along with the reason using the following methods:

  1. During the test using JavascriptExecutor
  2. After the test has completed using REST API

Mark test status from the test script using JavascriptExecutor

Your test script might contain a lot of assertions and you may choose to mark the status of the test based on any/all assertions, as per your test logic using JavascriptExecutor as shown below:

The arguments passed in the JavaScript method for setting the status and the corresponding reason for the test are status and reason.

  • status accepts either passed or failed as the value
  • reason accepts a value in string datatype

Following is the syntax for setting the status of the test in different languages.

JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"<passed/failed>\", \"reason\": \"<reason>\"}}");
driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>","reason": "<reason>"}}');
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"<passed/failed>\", \"reason\": \" <reason> \"}}");
$web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}' );
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
$driver->execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}');

Following is a sample test script showing the marking of test status and the associated reason:

package bs_automate;
import java.net.URL;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

public class bs_automate_class {
  public static final String USERNAME = "YOUR_USERNAME";
  public static final String AUTOMATE_KEY = "YOUR_ACCESS_KEY";
  public static final String URL = "https://" + USERNAME + ":" + AUTOMATE_KEY + "@hub-cloud.browserstack.com/wd/hub";
  public static void main(String[] args) throws Exception {

    // Input capabilities
    DesiredCapabilities caps = new DesiredCapabilities();
    caps.setCapability("browser", "Chrome");
    caps.setCapability("browser_version", "72.0");
    caps.setCapability("os", "Windows");
    caps.setCapability("os_version", "10");


    WebDriver driver = new RemoteWebDriver(new URL(URL), caps);
    JavascriptExecutor jse = (JavascriptExecutor)driver;

    // Searching for 'BrowserStack' on google.com
    driver.get("https://www.google.com");
    WebElement element = driver.findElement(By.name("q"));
    element.sendKeys("BrowserStack");
    element.submit();
    System.out.println(driver.getTitle());

    // Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
    if (driver.getTitle().equals("BrowserStack - Google Search")) {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Title matched!\"}}");
    }
    else {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Title not matched\"}}");
    }

    driver.quit();
  }
}
var webdriver = require('selenium-webdriver');

// Input capabilities
var capabilities = {
  'browserName': 'chrome',
  'browserVersion': '72.0',
  'os': 'windows',
  'os_version': '10'
}

var driver = new webdriver.Builder().
  usingServer('http://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub').
  withCapabilities(capabilities).
  build();
  
// Searching for 'BrowserStack' on google.com
driver.get('http://www.google.com').then(function(){
  driver.findElement(webdriver.By.name('q')).sendKeys('BrowserStack\n').then(function(){
    driver.getTitle().then(function(title) {
      console.log(title);

      // Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
      if(title === "BrowserStack - Google Search"){
      	driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Title matched!"}}');
      } else {
      	driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "Title not matched!"}}');
      }
	  driver.quit();
    });
  });
});
using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
namespace cs_testing
{
    class Program
    {
        static void Main(string[] args)
        {
            IWebDriver driver;

            // Input capabilities
            DesiredCapabilities capability = new DesiredCapabilities();
            capability.SetCapability("browser", "Chrome");
            capability.SetCapability("browser_version", "72.0");
            capability.SetCapability("os", "Windows");
            capability.SetCapability("os_version", "10");


            driver = new RemoteWebDriver(
                new Uri("http://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub/"), capability
            );

            // Searching for 'BrowserStack' on google.com
            driver.Navigate().GoToUrl("http://www.google.com");
            IWebElement query = driver.FindElement(By.Name("q"));
            query.SendKeys("BrowserStack");
            query.Submit();
            Console.WriteLine(driver.Title);

            // Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
            string str = "BrowserStack - Google Search";
            if (string.Equals(driver.Title, str))
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Title matched!\"}}");
            }
            else
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Title not matched \"}}");
            }
            driver.Quit();
        }
    }
}
<?php
  require_once('vendor/autoload.php');
  use Facebook\WebDriver\Remote\RemoteWebDriver;
  use Facebook\WebDriver\WebDriverBy;

  # Input capabilities
  $caps = array(
    "browserName"=> "chrome",
    "browserVersion"=>"72.0",
    "os"=>"windows",
    "os_version"=>"10",

  );

  $web_driver = RemoteWebDriver::create("https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub", $caps);

  # Searching for 'BrowserStack' on google.com
  $web_driver->get("http://google.com");
  $element = $web_driver->findElement(WebDriverBy::name("q"));
  if($element) {
      $element->sendKeys("BrowserStack");
      $element->submit();
  }
  print $web_driver->getTitle();

  # Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
  if ($web_driver->getTitle() == "BrowserStack - Google Search"){
      $web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
  }  else {
      $web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
  }

  $web_driver->quit();
?>
import requests
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

# Input capabilities
desired_cap = {
 'browserName': 'chrome',
 'browserVersion': '72.0',
 'os': 'windows',
 'os_version': '10'
}

driver = webdriver.Remote(
    command_executor=''https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub',
    desired_capabilities=desired_cap)

# Searching for 'BrowserStack' on google.com
driver.get("http://www.google.com")
if not "Google" in driver.title:
    raise Exception("Unable to load google page!")
elem = driver.find_element_by_name("q")
elem.send_keys("BrowserStack")
elem.submit()
print(driver.title)

# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if (driver.title=="BrowserStack - Google Search"):
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else:
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
driver.quit()
require "rubygems"
require "selenium-webdriver"

# Input capabilities
caps = Selenium::WebDriver::Remote::Capabilities.new
caps["os"] = "Windows"
caps["os_version"] = "10"
caps["browser"] = "Chrome"
caps["browser_version"] = "72.0"
caps["javascriptEnabled"] = "true"

driver = Selenium::WebDriver.for(:remote,
  :url => "http://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
  :desired_capabilities => caps)

# Searching for 'BrowserStack' on google.com
driver.navigate.to "http://www.google.com"
element = driver.find_element(:name, "q")
element.send_keys "BrowserStack"
element.submit

puts driver.title

# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if driver.title=="BrowserStack - Google Search"
  driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else
  driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
end

driver.quit
use Selenium::Remote::Driver;

# Input capabilities
my $extraCaps = {
  'browserName'=> 'chrome',
  'browserVersion'=> '72.0',
  'os'=> 'windows',
  'os_version'=>'10'
};

my $login = "YOUR_USERNAME";
my $key = "YOUR_ACCESS_KEY";
my $host = "$login:$key\@hub-cloud.browserstack.com";

my $driver = new Selenium::Remote::Driver('remote_server_addr' => $host,
  'port' => '80', 'extra_capabilities' => $extraCaps);

# Searching for 'BrowserStack' on google.com
$driver->get('http://www.google.com');
$driver->find_element('q','name')->send_keys("BrowserStack");
print ($driver->get_title());

# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if ($driver->get_title()=="BrowserStack - Google Search") {
	print ("correct");
	$driver->execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}');
} else {
	$driver->execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not match!"}}');
	print ("in-correct");
}
$driver->quit();

Mark test status after test completion using REST API

Suppose your test script has completed its run and now you want to mark the test on BrowserStack. You will not be able to follow the approach shown above in this scenario. But you can still mark your tests using our REST API endpoint for marking the status of test as passed or failed. You can also pass a reason to explain why you’re marking a test as failed (or passed).

The arguments passed in the REST API call for setting the status, and the corresponding reason of the test are status and reason.

  • status accepts either passed or failed as the value
  • reason accepts a value in string datatype

The following REST API can be used to achieve this:

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" -X PUT -H "Content-Type: application/json" \
-d "{\"status\":\"<new-status>\", \"reason\":\"<reason text>\"}" \
 https://api.browserstack.com/automate/sessions/<session-id>.json

Read here to know about how you can fetch the <session-id> of your session which you want to mark and use it in the REST API call.

We're sorry to hear that. Please share your feedback so we can do better






Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked





Thank you for your valuable feedback