Skip to main content

Mark tests as passed or failed

Mark your tests as passed or failed from within the test script or after the test has completed.

BrowserStack does not know whether your test’s assertions have passed or failed. Therefore, based on your test script’s assertions, you can explicitly inform BrowserStack whether your tests have passed or failed, and the test status can be marked accordingly. This document will help you in updating your test status.

Once you have executed the steps given below, the status and reason are visible on the test session details page of the App Automate dashboard as shown below:

Status reason passed or failed visible on the session details page of App Automate dashboard

You can mark the status of your test along with the reason using the following methods:

  1. From within test script using JavascriptExecutor
  2. After the test has completed using REST API

Mark test status from within test script using JavascriptExecutor

Your test script might contain a lot of assertions and you may choose to mark the status of the test based on any/all assertions, as per your test logic using a JavascriptExecutor during test execution.

The following arguments are required to be passed while using the JavascriptExecutor:

  • status accepts either passed or failed as the value
  • reason accepts a value in string datatype. This can be useful to record a reason for test failure.

Use the code snippet below to mark the status of the test using JavascriptExecutor.

JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"<passed/failed>\", \"reason\": \"<reason>\"}}");
driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>","reason": "<reason>"}}');
((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"<passed/failed>\", \"reason\": \" <reason> \"}}");
$driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}' );
driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')
caps["javascriptEnabled"] = "true" #Additionally, include this capability for JavaScript executors to work

driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"<passed/failed>", "reason": "<reason>"}}')

Following is a sample test script showing the marking of test status and the associated reason:

package android;

import java.net.URL;
import java.util.List;
import java.net.MalformedURLException;

import io.appium.java_client.MobileBy;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.android.AndroidElement;

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.remote.DesiredCapabilities;


public class BrowserStackSample {

  public static void main(String[] args) throws MalformedURLException, InterruptedException {
    
      DesiredCapabilities caps = new DesiredCapabilities();

      caps.setCapability("app", "bs://c700ce60cf13ae8ed97705a55b8e022f13c5827c");
      caps.setCapability("device", "Google Pixel 3");
      caps.setCapability("os_version", "9.0");

        AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
            new URL("http://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub.browserstack.com/wd/hub"), caps);

        JavascriptExecutor jse = (JavascriptExecutor)driver;
        

        AndroidElement searchElement = (AndroidElement) new WebDriverWait(driver, 30).until(
            ExpectedConditions.elementToBeClickable(
                MobileBy.AccessibilityId("Search Wikipedia")));
        searchElement.click();
    AndroidElement insertTextElement = (AndroidElement) new WebDriverWait(driver, 30).until(
             ExpectedConditions.elementToBeClickable(
                 MobileBy.id("org.wikipedia.alpha:id/search_src_text")));
        insertTextElement.sendKeys("BrowserStack");
        Thread.sleep(5000);
        List<AndroidElement> allProductsName = driver.findElementsByClassName(
            "android.widget.TextView");

    // Setting the status of test as 'passed' or 'failed' based on the condition: if results are found for the search
    if (allProductsName.size() > 0) {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \"passed\", \"reason\": \"Results found!\"}}");
    }
    else {
      jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \"Results not found\"}}");
    }

        driver.quit();
    
  }

}
let wd = require('wd');
let assert = require('assert');
let asserters = wd.asserters;

desiredCaps = {
  'app' : 'bs://c700ce60cf13ae8ed97705a55b8e022f13c5827c',
  'device' : 'Google Pixel 3',
  'os_version' : '9.0',
};

driver = wd.promiseRemote("http://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub");

driver.init(desiredCaps)
  .then(function () {
    return driver.waitForElementByAccessibilityId(
      'Search Wikipedia', asserters.isDisplayed 
      && asserters.isEnabled, 30000);
  })
  .then(function (searchElement) {
    return searchElement.click();
  })
  .then(function () {
    return driver.waitForElementById(
      'org.wikipedia.alpha:id/search_src_text', asserters.isDisplayed 
      && asserters.isEnabled, 30000);
  })
  .then(function (searchInput) {
    return searchInput.sendKeys("BrowserStack");
  })
  .then(function () {
    return driver.elementsByClassName('android.widget.TextView');    
  })
  .then(function (search_results) {
 
    // Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
    if(search_results.length > 0){
      return driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed","reason": "Results found!"}}');
    } else {
      driver.execute('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed","reason": "No results available!"}}');
    
     }
  })
  .fin(function() { 
    return driver.quit(); 
  })
  .done();
using System;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Android;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
using System.Collections.Generic;
using OpenQA.Selenium;

namespace csharp_appium_first_test_browserstack
{
    class Program
    {
        static void Main(string[] args)
        {
            AppiumOptions caps = new AppiumOptions();

            // Set URL of the application under test
            caps.AddAdditionalCapability("app", "bs://c700ce60cf13ae8ed97705a55b8e022f13c5827c");
            caps.AddAdditionalCapability("device", "Google Pixel 3");
            caps.AddAdditionalCapability("os_version", "9.0");
            caps.PlatformName = "Android";

            AndroidDriver<AndroidElement> driver = new AndroidDriver<AndroidElement>(
                    new Uri("http://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub"), caps);

            AndroidElement searchElement = (AndroidElement)new WebDriverWait(
                driver, TimeSpan.FromSeconds(30)).Until(
                SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
                    MobileBy.AccessibilityId("Search Wikipedia"))
            );
            searchElement.Click();
            AndroidElement insertTextElement = (AndroidElement)new WebDriverWait(
                driver, TimeSpan.FromSeconds(30)).Until(
                SeleniumExtras.WaitHelpers.ExpectedConditions.ElementToBeClickable(
                    MobileBy.Id("org.wikipedia.alpha:id/search_src_text"))
            );
            insertTextElement.SendKeys("BrowserStack");
            System.Threading.Thread.Sleep(5000);

            IReadOnlyList<AndroidElement> allTextViewElements =
                driver.FindElementsByClassName("android.widget.TextView");

            // Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search

            if ((allTextViewElements.Count > 0))
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"passed\", \"reason\": \" Results found! \"}}");
            }
            else
            {
                ((IJavaScriptExecutor)driver).ExecuteScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\":\"failed\", \"reason\": \" Results not found!\"}}");
            }

            driver.Quit();
        }
    }
}
<?php
  require_once('vendor/autoload.php');
    use Facebook\WebDriver\Remote\DesiredCapabilities;
    use Facebook\WebDriver\Remote\RemoteWebDriver;
    use Facebook\WebDriver\WebDriverBy;

    $caps = array(
        "app" => "bs://c700ce60cf13ae8ed97705a55b8e022f13c5827c",
        "device" => "Google Pixel 3",
        "name" => "Bstack-[Php] Sample Test",
        "build" => "Bstack-[Php] Sample Build"
    );

    $driver = RemoteWebDriver::create("https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub", $caps);


  
    $driver->getTitle();

  # Setting the status of test as 'passed' or 'failed' based on the condition; if title of the web page matches 'Wikipedia'
  if ($driver->getTitle() == "Wikipedia"){
      $driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Title matched!"}}' );
  }  else {
      $driver->executeScript('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "Title not matched!"}}');
  }

  $driver->quit();
?>
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

desired_cap = {
   
    "app" : "bs://c700ce60cf13ae8ed97705a55b8e022f13c5827c",
    "device" : "Google Pixel 3",
    "os_version" : "9.0",
}

driver = webdriver.Remote(
    command_executor="http://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub", 
    desired_capabilities=desired_cap
)


search_element = WebDriverWait(driver, 30).until(
    EC.element_to_be_clickable((MobileBy.ACCESSIBILITY_ID, "Search Wikipedia"))
)
search_element.click()
search_input = WebDriverWait(driver, 30).until(
    EC.element_to_be_clickable((MobileBy.ID, "org.wikipedia.alpha:id/search_src_text"))
)
search_input.send_keys("BrowserStack")
time.sleep(5)

# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search
search_results = driver.find_elements_by_class_name("android.widget.TextView")
if (len(search_results) > 0):
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found!"}}')
else:
	driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
driver.quit()
require 'rubygems'
require 'appium_lib'


caps = {}
caps['app'] = 'bs://c700ce60cf13ae8ed97705a55b8e022f13c5827c'
caps['device'] = 'Google Pixel 3'
caps['os_version'] = '9.0'
caps['platformName'] = 'android'
caps['javascriptEnabled'] = 'true' #include this capability for JavaScript executors to work


appium_driver = Appium::Driver.new({
    'caps' => caps,
    'appium_lib' => {
        :server_url => "https://#YOUR_USERNAME:#YOUR_ACCESSKEY@hub-cloud.browserstack.com/wd/hub"
    }}, true)
driver = appium_driver.start_driver

wait = Selenium::WebDriver::Wait.new(:timeout => 30)
wait.until { driver.find_element(:accessibility_id, "Search Wikipedia").displayed? }
element = driver.find_element(:accessibility_id, "Search Wikipedia")
element.click

wait.until { driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text").displayed? }
search_box = driver.find_element(:id, "org.wikipedia.alpha:id/search_src_text")
search_box.send_keys("BrowserStack")
sleep 5

# Setting the status of test as 'passed' or 'failed' based on the condition if results are found for the search

results = driver.find_elements(:class, "android.widget.TextView")
if results.count > 0
    driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"passed", "reason": "Results found"}}')
else
   driver.execute_script('browserstack_executor: {"action": "setSessionStatus", "arguments": {"status":"failed", "reason": "No results found"}}')
end

driver.quit

Mark test status after test has completed using REST API

You can also mark a test session’s status as passed or failed using our REST API. This is particularly useful when a test successfully completed on BrowserStack (i.e. Completed status) but there was a failed assertion in the test case.

The following arguments need to be passed in the REST API call for setting the status, and its corresponding reason:

  • status accepts either passed or failed as the value
  • reason accepts a value in string datatype. This can be useful to record a reason for test failure.

Here is a REST API request to update the status and reason for an example test session :

curl -u "YOUR_USERNAME:YOUR_ACCESS_KEY" \
-X PUT "https://api-cloud.browserstack.com/app-automate/sessions/f97f02bf39d592f5fc349cee419294fdfb7593a2.json" \
-H "Content-Type: application/json" \
-d '{"status":"failed", "reason":"Element not found on the login page"}'

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback

Is this page helping you?

Yes
No

We're sorry to hear that. Please share your feedback so we can do better

Contact our Support team for immediate help while we work on improving our docs.

We're continuously improving our docs. We'd love to know what you liked






Thank you for your valuable feedback!

Talk to an Expert
Talk to an Expert