Mark tests as passed or failed
Mark your tests as passed or failed from within the test script or after the test has completed.
BrowserStack does not know whether your test’s assertions have passed or failed. Therefore, based on your test script’s assertions, you can explicitly inform BrowserStack whether your tests have passed or failed, and the test status can be marked accordingly. This document will help you in updating your test status.
Once you have executed the steps given below, the status and reason are visible on the test session details page of the App Automate dashboard as shown below:
You can mark the status of your test along with the reason using the following methods:
Mark test status from within test script using JavascriptExecutor
Your test script might contain a lot of assertions and you may choose to mark the status of the test based on any/all assertions, as per your test logic using a JavascriptExecutor during test execution.
The following arguments are required to be passed while using the JavascriptExecutor:
-
status
accepts eitherpassed
orfailed
as the value -
reason
accepts a value in string datatype. This can be useful to record a reason for test failure.
Use the code snippet below to mark the status of the test using JavascriptExecutor.
JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"<passed/failed>\", \"reason\": \"<reason>\"}}");
driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>","reason": "<reason>"}}');
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"<passed/failed>\", \"reason\": \" <reason> \"}}");
$driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}' );
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
caps["javascriptEnabled"] = "true" #Additionally, include this capability for JavaScript executors to work
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
If you are using BrowserStack SDK, setSessionStatus
is managed autonomously by the SDK. However, this is applicable only for language frameworks. For vanilla flavours, Session status needs to be set manually using JavaScript Executor.
Following is a sample test script shows the marking of test status and the associated reason:
package android;
import java.net.URL;
import java.util.List;
import java.net.MalformedURLException;
import io.appium.java_client.MobileBy;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.AndroidElement;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.remote.DesiredCapabilities;
public class BrowserStackSample {
public static void main(String[] args) throws MalformedURLException, InterruptedException {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("platformName", "android");
capabilities.setCapability("platformVersion", "9.0");
capabilities.setCapability("deviceName", "Google Pixel 3");
capabilities.setCapability("app", "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32");
AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
new URL("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub.browserstack.com/wd/hub"), capabilities);
JavascriptExecutor jse = (JavascriptExecutor)driver;
AndroidElement searchElement = (AndroidElement) new WebDriverWait(driver, 30).until(
ExpectedConditions.elementToBeClickable(
MobileBy.AccessibilityId("Search Wikipedia")));
searchElement.click();
AndroidElement insertTextElement = (AndroidElement) new WebDriverWait(driver, 30).until(
ExpectedConditions.elementToBeClickable(
MobileBy.id("org.wikipedia.alpha:id/search_src_text")));
insertTextElement.sendKeys("BrowserStack");
Thread.sleep(5000);
List<AndroidElement> allProductsName = driver.findElementsByClassName(
"android.widget.TextView");
// Setting the status of test as 'passed' or 'failed' based on the condition: if results are found for the search
if (allProductsName.size() > 0) {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Results found!\"}}");
}
else {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Results not found\"}}");
}
driver.quit();
}
}
let wd = require('wd');
let assert = require('assert');
let asserters = wd.asserters;
var Caps = {
"platformName" : "android",
"platformVersion" : "9.0",
"deviceName" : "Google Pixel 3",
"app" : "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
};
driver = wd.promiseRemote("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub");
driver.init(desiredCaps)
.then(function () {
return driver.waitForElementByAccessibilityId(
'Search Wikipedia', asserters.isDisplayed
&& asserters.isEnabled, 30000);
})
.then(function (searchElement) {
return searchElement.click();
})
.then(function () {
return driver.waitForElementById(
'org.wikipedia.alpha:id/search_src_text', asserters.isDisplayed
&& asserters.isEnabled, 30000);
})
.then(function (searchInput) {
return searchInput.sendKeys("BrowserStack");
})
.then(function () {
return driver.elementsByClassName('android.widget.TextView');
})
.then(function (search_results) {
// Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
if(search_results.length > 0){
return driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Results found!"}}');
} else {
driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "No results available!"}}');
}
})
.fin(function() {
return driver.quit();
})
.done();
using System;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Android;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
using System.Collections.Generic;
using OpenQA.Selenium;
namespace csharp_appium_first_test_browserstack
{
class Program
{
static void Main(string[] args)
{
AppiumOptions capabilities = new AppiumOptions();
// Set URL of the application under test
capabilities.AddAdditionalCapability("bstack:options", browserstackOptions);
capabilities.AddAdditionalCapability("platformName", "android");
capabilities.AddAdditionalCapability("platformVersion", "9.0");
capabilities.AddAdditionalCapability("appium:deviceName", "Google Pixel 3");
capabilities.AddAdditionalCapability("appium:app", "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32");
AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
new Uri("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub"), capabilities);
AndroidElement searchElement = (AndroidElement)new WebDriverWait(
driver, TimeSpan.FromSeconds(30)).Until(
SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
MobileBy.AccessibilityId("Search Wikipedia"))
);
searchElement.Click();
AndroidElement insertTextElement = (AndroidElement)new WebDriverWait(
driver, TimeSpan.FromSeconds(30)).Until(
SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
MobileBy.Id("org.wikipedia.alpha:id/search_src_text"))
);
insertTextElement.SendKeys("BrowserStack");
System.Threading.Thread.Sleep(5000);
IReadOnlyList<AndroidElement> allTextViewElements =
driver.FindElementsByClassName("android.widget.TextView");
// Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
if ((allTextViewElements.Count > 0))
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Results found! \"}}");
}
else
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Results not found!\"}}");
}
driver.Quit();
}
}
}
<?php
require_once('vendor/autoload.php');
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\WebDriverBy;
$caps = array(
"platformName" => "android",
"platformVersion" => "9.0",
"deviceName" => "Google Pixel 3",
"app" => "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
)
$driver = RemoteWebDriver::create("https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub", $caps);
$driver->getTitle();
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'Wikipedia'
if ($driver->getTitle() == "Wikipedia"){
$driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
} else {
$driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
}
$driver->quit();
?>
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
desired_cap = {
"platformName" : "android",
"platformVersion" : "9.0",
"deviceName" : "Google Pixel 3",
"app" : "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
}
driver = webdriver.Remote(
command_executor="https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub",
desired_capabilities=desired_cap
)
search_element = WebDriverWait(driver, 30).until(
EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Search Wikipedia"))
)
search_element.click()
search_input = WebDriverWait(driver, 30).until(
EC.element_to_be_clickable((MobileBy.ID, "org.wikipedia.alpha:id/search_src_text"))
)
search_input.send_keys("BrowserStack")
time.sleep(5)
# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
search_results = driver.find_elements_by_class_name("android.widget.TextView")
if (len(search_results) > 0):
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found!"}}')
else:
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
driver.quit()
require 'rubygems'
require 'appium_lib'
caps = {
"platformName" => "android",
"platformVersion" => "9.0",
"deviceName" => "Google Pixel 3",
"app" => "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
}
appium_driver = Appium::Driver.new({
'caps' => caps,
'appium_lib' => {
:server_url => "https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub"
}}, true)
driver = appium_driver.start_driver
wait = Selenium::WebDriver::Wait.new(:timeout => 30)
wait.until { driver.find_element(:accessibility_id, "Search Wikipedia").displayed? }
element = driver.find_element(:accessibility_id, "Search Wikipedia")
element.click
wait.until { driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text").displayed? }
search_box = driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text")
search_box.send_keys("BrowserStack")
sleep 5
# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
results = driver.find_elements(:class, "android.widget.TextView")
if results.count > 0
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found"}}')
else
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
end
driver.quit
package android;
import java.net.URL;
import java.util.List;
import java.net.MalformedURLException;
import io.appium.java_client.MobileBy;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.AndroidElement;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.remote.DesiredCapabilities;
public class BrowserStackSample {
public static void main(String[] args) throws MalformedURLException, InterruptedException {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("os_version", "9.0");
capabilities.setCapability("device", "Google Pixel 3");
capabilities.setCapability("app", "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32");
AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
new URL("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub.browserstack.com/wd/hub"), capabilities);
JavascriptExecutor jse = (JavascriptExecutor)driver;
AndroidElement searchElement = (AndroidElement) new WebDriverWait(driver, 30).until(
ExpectedConditions.elementToBeClickable(
MobileBy.AccessibilityId("Search Wikipedia")));
searchElement.click();
AndroidElement insertTextElement = (AndroidElement) new WebDriverWait(driver, 30).until(
ExpectedConditions.elementToBeClickable(
MobileBy.id("org.wikipedia.alpha:id/search_src_text")));
insertTextElement.sendKeys("BrowserStack");
Thread.sleep(5000);
List<AndroidElement> allProductsName = driver.findElementsByClassName(
"android.widget.TextView");
// Setting the status of test as 'passed' or 'failed' based on the condition: if results are found for the search
if (allProductsName.size() > 0) {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Results found!\"}}");
}
else {
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Results not found\"}}");
}
driver.quit();
}
}
let wd = require('wd');
let assert = require('assert');
let asserters = wd.asserters;
var capabilities = {
"os_version" : "9.0",
"device" : "Google Pixel 3",
"app" : "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32",
}
driver = wd.promiseRemote("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub");
driver.init(desiredCaps)
.then(function () {
return driver.waitForElementByAccessibilityId(
'Search Wikipedia', asserters.isDisplayed
&& asserters.isEnabled, 30000);
})
.then(function (searchElement) {
return searchElement.click();
})
.then(function () {
return driver.waitForElementById(
'org.wikipedia.alpha:id/search_src_text', asserters.isDisplayed
&& asserters.isEnabled, 30000);
})
.then(function (searchInput) {
return searchInput.sendKeys("BrowserStack");
})
.then(function () {
return driver.elementsByClassName('android.widget.TextView');
})
.then(function (search_results) {
// Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
if(search_results.length > 0){
return driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Results found!"}}');
} else {
driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "No results available!"}}');
}
})
.fin(function() {
return driver.quit();
})
.done();
using System;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Android;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
using System.Collections.Generic;
using OpenQA.Selenium;
namespace csharp_appium_first_test_browserstack
{
class Program
{
static void Main(string[] args)
{
AppiumOptions capabilities = new AppiumOptions();
capabilities.AddAdditionalCapability("os_version", "9.0");
capabilities.AddAdditionalCapability("device", "Google Pixel 3");
capabilities.AddAdditionalCapability("app", "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32");
capabilities.AddAdditionalCapability("browserstack.user", "YOUR_USERNAME");
capabilities.AddAdditionalCapability("browserstack.key", "YOUR_ACCESS_KEY");
AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
new Uri("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub"), capabilities);
AndroidElement searchElement = (AndroidElement)new WebDriverWait(
driver, TimeSpan.FromSeconds(30)).Until(
SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
MobileBy.AccessibilityId("Search Wikipedia"))
);
searchElement.Click();
AndroidElement insertTextElement = (AndroidElement)new WebDriverWait(
driver, TimeSpan.FromSeconds(30)).Until(
SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
MobileBy.Id("org.wikipedia.alpha:id/search_src_text"))
);
insertTextElement.SendKeys("BrowserStack");
System.Threading.Thread.Sleep(5000);
IReadOnlyList<AndroidElement> allTextViewElements =
driver.FindElementsByClassName("android.widget.TextView");
// Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
if ((allTextViewElements.Count > 0))
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Results found! \"}}");
}
else
{
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Results not found!\"}}");
}
driver.Quit();
}
}
}
<?php
require_once('vendor/autoload.php');
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\WebDriverBy;
$caps = array(
"os_version" => "9.0",
"device" => "Google Pixel 3",
"app" => "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32"
);
$driver = RemoteWebDriver::create("https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub", $caps);
$driver->getTitle();
# Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'Wikipedia'
if ($driver->getTitle() == "Wikipedia"){
$driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
} else {
$driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
}
$driver->quit();
?>
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
desired_cap = {
"os_version" : "9.0",
"device" : "Google Pixel 3",
"app" : "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32"
}
driver = webdriver.Remote(
command_executor="https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub",
desired_capabilities=desired_cap
)
search_element = WebDriverWait(driver, 30).until(
EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Search Wikipedia"))
)
search_element.click()
search_input = WebDriverWait(driver, 30).until(
EC.element_to_be_clickable((MobileBy.ID, "org.wikipedia.alpha:id/search_src_text"))
)
search_input.send_keys("BrowserStack")
time.sleep(5)
# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
search_results = driver.find_elements_by_class_name("android.widget.TextView")
if (len(search_results) > 0):
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found!"}}')
else:
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
driver.quit()
require 'rubygems'
require 'appium_lib'
caps = Selenium::WebDriver::Remote::Capabilities.new
caps["os_version"] = "9.0"
caps["device"] = "Google Pixel 3"
caps["app"] = "bs://5a96eae8cd61b93bb4bcdb287bbe3e9ba4f11a32"
appium_driver = Appium::Driver.new({
'caps' => caps,
'appium_lib' => {
:server_url => "https://#YOUR_USERNAME:#YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub"
}}, true)
driver = appium_driver.start_driver
wait = Selenium::WebDriver::Wait.new(:timeout => 30)
wait.until { driver.find_element(:accessibility_id, "Search Wikipedia").displayed? }
element = driver.find_element(:accessibility_id, "Search Wikipedia")
element.click
wait.until { driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text").displayed? }
search_box = driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text")
search_box.send_keys("BrowserStack")
sleep 5
# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
results = driver.find_elements(:class, "android.widget.TextView")
if results.count > 0
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found"}}')
else
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
end
driver.quit
Mark test status after test has completed using REST API
You can also mark a test session’s status as passed
or failed
using our REST API. This is particularly useful when a test successfully completed on BrowserStack (i.e. Completed
status) but there was a failed assertion in the test case.
The following arguments need to be passed in the REST API call for setting the status, and its corresponding reason:
-
status
accepts eitherpassed
orfailed
as the value -
reason
accepts a value in string datatype. This can be useful to record a reason for test failure.
Here is a REST API request to update the status
and reason
for an example test session :
curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'
Character Limits
- A maximum of 256 characters can be passed as input value for
reason
argument. - Escape sequences such as
\n
,\t
,\b
,\"
,\r
are not accepted as valid input forreason
argument.
Related topics
Check this page for a list of various JavaScript Executors that BrowserStack offers.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!