Mark tests as passed or failed
Mark your Automate tests as passed or failed from within the test script or after the test has completed.
BrowserStack cannot identify if your test’s assertions have passed
or failed
. Therefore, based on the assertions in your test script, you have to explicitly instruct BrowserStack whether your tests have passed or failed.
This guide will provide detailed information on how to mark your tests as passed
or failed
.
Once you have completed your test run, you will be able to view the status of your tests on the Automate Dashboard as shown below:
You can mark the status of your test along with the reason using the following methods:
You can also edit the test status and reason from the Automate Dashboard, check out how to edit your test session details.
Mark test status from the test script using JavascriptExecutor
Your test script can contain many assertions. You may choose to mark the status of the test based on these assertions using JavascriptExecutor.
Used the following arguments in the JavaScript method to set the status and the corresponding reason for the test:
Argument | Description | Accepted values |
---|---|---|
status |
Status of the test | A string. passed or failed
|
reason |
Test Pass or Fail reason | A string. |
Use the following code snippet to set the status of the test:
final JavascriptExecutor jse = (JavascriptExecutor) driver;
JSONObject executorObject = new JSONObject();
JSONObject argumentsObject = new JSONObject();
argumentsObject.put("status", "<passed/failed>");
argumentsObject.put("reason", "<reason>");
executorObject.put("action", "setSessionStatus");
executorObject.put("arguments", argumentsObject);
jse.executeScript(String.format("browserstack_executor: %s", executorObject));
const executorConfig = {
"action": "setSessionStatus",
"arguments": {
"status": "<passed/failed>",
"reason": "<reason>"
}
};
await driver.executeScript('browserstack_executor: ' + JSON.stringify(executorConfig));
using System.Text.Json.Nodes;
IJavaScriptExecutor jse = (IJavaScriptExecutor)driver;
JsonObject executorObject = new JsonObject();
JsonObject argumentsObject = new JsonObject();
argumentsObject.Add("status", "<passed/failed>");
argumentsObject.Add("reason", "<reason>");
executorObject.Add("action", "setSessionStatus");
executorObject.Add("arguments", argumentsObject);
jse.ExecuteScript("browserstack_executor: " + executorObject.ToString());
$web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}' );
import json
executor_object = {
'action': 'setSessionStatus',
'arguments': {
'status': "<passed/failed>",
'reason': "<reason>"
}
}
browserstack_executor = 'browserstack_executor: {}'.format(json.dumps(executor_object))
driver.execute_script(browserstack_executor)
executor_config = {
action: "setSessionStatus",
arguments: {
status: "<passed/failed>",
reason: "<reason>"
}
}
json = executor_config.to_json
script = 'browserstack_executor: ' + json
driver.execute_script(script)
If you are using BrowserStack SDK, setSessionStatus
is managed autonomously by the SDK. However, this is applicable only for language frameworks. For vanilla flavours, Session status needs to be set manually using JavaScript Executor.
The following sample test script shows the test status and the associated reason:
package bs_automate;
import java.net.URL;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.MutableCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
public class bs_automate_class {
public static final String USERNAME = "YOUR_USERNAME";
public static final String AUTOMATE_KEY = "YOUR_ACCESS_KEY";
public static final String URL = "https://" + USERNAME + ":" + AUTOMATE_KEY + "@hub-cloud.browserstack.com/wd/hub";
public static void main(String[] args) throws Exception {
// Input capabilities
MutableCapabilities capabilities = new MutableCapabilities();
capabilities.setCapability("browserName", "Chrome");
capabilities.setCapability("browserVersion", "103.0");
HashMap<String, Object> browserstackOptions = new HashMap<String, Object>();
browserstackOptions.put("os", "Windows");
browserstackOptions.put("osVersion", "10");
browserstackOptions.put("resolution", "1920x1080");
browserstackOptions.put("projectName", "Mark test as pass fail using JS Executor"); // test name
browserstackOptions.put("buildName", "Sample Build"); // CI/CD job or build name
browserstackOptions.put("seleniumVersion", "4.0.0");
capabilities.setCapability("bstack:options", browserstackOptions);
// Searching for 'BrowserStack' on google.com
driver.get("https://www.google.com");
WebElement element = driver.findElement(By.name("q"));
element.sendKeys("BrowserStack");
element.submit();
System.out.println(driver.getTitle());
// Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if (driver.getTitle().equals("BrowserStack - Google Search")) {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Title matched!\"}}");
}
else {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Title not matched\"}}");
}
driver.quit();
}
}
var webdriver = require('selenium-webdriver');
// Input capabilities
var capabilities = {
'bstack:options' : {
"os" : "Windows",
"osVersion" : "10",
"resolution" : "1920x1080",
"projectName" : "Mark test as pass fail using JS Executor", // test name
"buildName" : "Sample Build", // CI/CD job or build name
"seleniumVersion" : "4.0.0",
"userName" : "YOUR_USERNAME",
"accessKey" : "YOUR_ACCESS_KEY",
},
"browserName" : "Chrome",
"browserVersion" : "103.0",
}
var driver = new webdriver.Builder().
usingServer('https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub').
withCapabilities(capabilities).
build();
// Searching for 'BrowserStack' on google.com
driver.get('https://www.google.com').then(function(){
driver.findElement(webdriver.By.name('q')).sendKeys('BrowserStack\n').then(function(){
driver.getTitle().then(function(title) {
console.log(title);
// Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if(title === "BrowserStack - Google Search"){
driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Title matched!"}}');
} else {
driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "Title not matched!"}}');
}
driver.quit();
});
});
});
using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
namespace cs_testing
{
class Program
{
static void Main(string[] args)
{
IWebDriver driver;
// Input capabilities
ChromeOptions capabilities = new ChromeOptions();
capabilities.BrowserVersion = "103.0";
Dictionary<string, object> browserstackOptions = new Dictionary<string, object>();
browserstackOptions.Add("os", "Windows");
browserstackOptions.Add("osVersion", "10");
browserstackOptions.Add("resolution", "1920x1080");
browserstackOptions.Add("projectName", "Mark test as pass fail using JS Executor"); // test name
browserstackOptions.Add("buildName", "Sample Build"); // CI/CD job or build name
browserstackOptions.Add("seleniumVersion", "4.0.0");
browserstackOptions.Add("userName", "YOUR_USERNAME");
browserstackOptions.Add("accessKey", "YOUR_ACCESS_KEY");
capabilities.AddAdditionalOption("bstack:options", browserstackOptions);
// Searching for 'BrowserStack' on google.com
driver.Navigate().GoToUrl("https://www.google.com");
IWebElement query = driver.FindElement(By.Name("q"));
query.SendKeys("BrowserStack");
query.Submit();
Console.WriteLine(driver.Title);
// Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
string str = "BrowserStack - Google Search";
if (string.Equals(driver.Title, str))
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Title matched!\"}}");
}
else
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Title not matched \"}}");
}
driver.Quit();
}
}
}
<?php
require_once('vendor/autoload.php');
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\WebDriverBy;
# Input capabilities
$caps = array(
'bstack:options' => array(
"os" => "Windows",
"osVersion" => "10",
"resolution" => "1920x1080",
"projectName" => "Mark test as pass fail using JS Executor", #test name
"buildName" => "Sample Build", #CI/CD job or build name
"seleniumVersion" => "4.0.0",
),
"browserName" => "Chrome",
"browserVersion" => "103.0",
)
$web_driver = RemoteWebDriver::create("https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub", $caps);
# Searching for 'BrowserStack' on google.com
$web_driver->get("https://google.com");
$element = $web_driver->findElement(WebDriverBy::name("q"));
if($element) {
$element->sendKeys("BrowserStack");
$element->submit();
}
print $web_driver->getTitle();
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if ($web_driver->getTitle() == "BrowserStack - Google Search"){
$web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
} else {
$web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
}
$web_driver->quit();
?>
import requests
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
# Input capabilities
desired_cap = {
'bstack:options' : {
"os" : "Windows",
"osVersion" : "10",
"resolution" : "1920x1080",
"projectName" : "Mark test as pass fail using JS Executor", #test name
"buildName" : "Sample Build", #CI/CD job or build name
"seleniumVersion" : "4.0.0",
},
"browserName" : "Chrome",
"browserVersion" : "103.0",
}
driver = webdriver.Remote(
command_executor='https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub',
desired_capabilities=desired_cap)
# Searching for 'BrowserStack' on google.com
driver.get("https://www.google.com")
if not "Google" in driver.title:
raise Exception("Unable to load google page!")
elem = driver.find_element_by_name("q")
elem.send_keys("BrowserStack")
elem.submit()
print(driver.title)
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if (driver.title=="BrowserStack - Google Search"):
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else:
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
driver.quit()
require "rubygems"
require "selenium-webdriver"
# Input capabilities
capabilities = {
'bstack:options' => {
"os" => "Windows",
"osVersion" => "10",
"resolution" => "1920x1080",
"projectName" => "Mark test as pass fail using JS Executor", #test name
"buildName" => "Sample Build", #CI/CD job or build name
"seleniumVersion" => "4.0.0",
},
"browserName" => "Chrome",
"browserVersion" => "103.0",
}
driver = Selenium::WebDriver.for(:remote,
:url => "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
:desired_capabilities => caps)
# Searching for 'BrowserStack' on google.com
driver.navigate.to "https://www.google.com"
element = driver.find_element(:name, "q")
element.send_keys "BrowserStack"
element.submit
puts driver.title
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if driver.title=="BrowserStack - Google Search"
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
end
driver.quit
package bs_automate;
import java.net.URL;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
public class bs_automate_class {
public static final String USERNAME = "YOUR_USERNAME";
public static final String AUTOMATE_KEY = "YOUR_ACCESS_KEY";
public static final String URL = "https://" + USERNAME + ":" + AUTOMATE_KEY + "@hub-cloud.browserstack.com/wd/hub";
public static void main(String[] args) throws Exception {
// Input capabilities
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("browser", "Chrome");
caps.setCapability("browser_version", "72.0");
caps.setCapability("os", "Windows");
caps.setCapability("os_version", "10");
WebDriver driver = new RemoteWebDriver(new URL(URL), caps);
JavascriptExecutor jse = (JavascriptExecutor)driver;
// Searching for 'BrowserStack' on google.com
driver.get("https://www.google.com");
WebElement element = driver.findElement(By.name("q"));
element.sendKeys("BrowserStack");
element.submit();
System.out.println(driver.getTitle());
// Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if (driver.getTitle().equals("BrowserStack - Google Search")) {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Title matched!\"}}");
}
else {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Title not matched\"}}");
}
driver.quit();
}
}
var webdriver = require('selenium-webdriver');
// Input capabilities
var capabilities = {
'browserName': 'chrome',
'browserVersion': '72.0',
'os': 'windows',
'os_version': '10'
}
var driver = new webdriver.Builder().
usingServer('https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub').
withCapabilities(capabilities).
build();
// Searching for 'BrowserStack' on google.com
driver.get('https://www.google.com').then(function(){
driver.findElement(webdriver.By.name('q')).sendKeys('BrowserStack\n').then(function(){
driver.getTitle().then(function(title) {
console.log(title);
// Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if(title === "BrowserStack - Google Search"){
driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Title matched!"}}');
} else {
driver.executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "Title not matched!"}}');
}
driver.quit();
});
});
});
using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
namespace cs_testing
{
class Program
{
static void Main(string[] args)
{
IWebDriver driver;
// Input capabilities
OpenQA.Selenium.Chrome.ChromeOptions capability = new OpenQA.Selenium.Chrome.ChromeOptions();
capability.AddAdditionalCapability("os_version", "10", true);
capability.AddAdditionalCapability("resolution", "1920x1080", true);
capability.AddAdditionalCapability("browser", "Chrome", true);
capability.AddAdditionalCapability("browser_version", "latest", true);
capability.AddAdditionalCapability("os", "Windows", true);
capability.AddAdditionalCapability("name", "Mark test as pass fail using JS Executor", true); // test name
capability.AddAdditionalCapability("build", "C-sharp Sample Build", true); // CI/CD job or build name
capability.AddAdditionalCapability("browserstack.user", "YOUR_USERNAME", true);
capability.AddAdditionalCapability("browserstack.key", "YOUR_ACCESS_KEY", true);
driver = new RemoteWebDriver(
new Uri("https://hub-cloud.browserstack.com/wd/hub/"), capability
);
// Searching for 'BrowserStack' on google.com
driver.Navigate().GoToUrl("https://www.google.com");
IWebElement query = driver.FindElement(By.Name("q"));
query.SendKeys("BrowserStack");
query.Submit();
Console.WriteLine(driver.Title);
// Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
string str = "BrowserStack - Google Search";
if (string.Equals(driver.Title, str))
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Title matched!\"}}");
}
else
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Title not matched \"}}");
}
driver.Quit();
}
}
}
<?php
require_once('vendor/autoload.php');
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\WebDriverBy;
# Input capabilities
$caps = array(
"browserName"=> "chrome",
"browserVersion"=>"72.0",
"os"=>"windows",
"os_version"=>"10",
);
$web_driver = RemoteWebDriver::create("https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub", $caps);
# Searching for 'BrowserStack' on google.com
$web_driver->get("https://google.com");
$element = $web_driver->findElement(WebDriverBy::name("q"));
if($element) {
$element->sendKeys("BrowserStack");
$element->submit();
}
print $web_driver->getTitle();
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if ($web_driver->getTitle() == "BrowserStack - Google Search"){
$web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
} else {
$web_driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
}
$web_driver->quit();
?>
import requests
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
# Input capabilities
desired_cap = {
'browserName': 'chrome',
'browserVersion': '72.0',
'os': 'windows',
'os_version': '10'
}
driver = webdriver.Remote(
command_executor='https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub',
desired_capabilities=desired_cap)
# Searching for 'BrowserStack' on google.com
driver.get("https://www.google.com")
if not "Google" in driver.title:
raise Exception("Unable to load google page!")
elem = driver.find_element_by_name("q")
elem.send_keys("BrowserStack")
elem.submit()
print(driver.title)
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if (driver.title=="BrowserStack - Google Search"):
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else:
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
driver.quit()
require "rubygems"
require "selenium-webdriver"
# Input capabilities
caps = Selenium::WebDriver::Remote::Capabilities.new
caps["os"] = "Windows"
caps["os_version"] = "10"
caps["browser"] = "Chrome"
caps["browser_version"] = "72.0"
caps["javascriptEnabled"] = "true"
driver = Selenium::WebDriver.for(:remote,
:url => "https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
:desired_capabilities => caps)
# Searching for 'BrowserStack' on google.com
driver.navigate.to "https://www.google.com"
element = driver.find_element(:name, "q")
element.send_keys "BrowserStack"
element.submit
puts driver.title
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'BrowserStack - Google Search'
if driver.title=="BrowserStack - Google Search"
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}')
else
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched"}}')
end
driver.quit
Mark test status after test completion using REST API
You can also mark your test as as passed
or failed
on BrowserStack after the test script has completed its run, however, you will not be able to follow the above-mentioned approach.
In such cases, it is recommended that you mark your tests using our REST API endpoint to mark the status of test as passed
or failed
. You can also provide a reason to explain why you’re marking a test as failed (or passed).
Use the following arguments in the REST API call to set the test status, and the corresponding reason:
Argument | Description | Accepted values |
---|---|---|
status |
Status of the test | A string. passed or failed
|
reason |
Test Pass or Fail reason | A string. |
The following REST API can be used to achieve this:
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" -X PUT -H "Content-Type: application/json" \
-d "{\"status\":\"<new-status>\", \"reason\":\"<reason text>\"}" \
https://api.browserstack.com/automate/sessions/<session-id>.json
Read more about how to fetch the <session-id>
of your session which you want to mark and use it in the REST API call.
Related topics
Check this page for a list of various JavaScript Executors that BrowserStack offers.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!