How to report bugs in Appium UI Testing?

The backbone of Appium is derived from Selenium. But the main difference is that Selenium can automate only web and desktop browsers. On the other side, Appium has the same features as Selenium with additional support for mobile app automating capability. Their origin is from the same protocol named- JSON Wire Protocol. They use the same JSON wire endpoint.

GET /wd/hub/session/:sessionId/log/types
POST /wd/hub/session/:sessionId/log

Also, they use the same WebDriver, which is Selenium WebDriver, and Desired Capabilities to interact with Android and iOS applications.

So, both of the testing with Appium and Selenium are the same. Thus they depend upon the same test reporting tools like TestNG, JUnit, etc.

Extent Reporting in Appium UI testing

Though you can generate in-built reports by JUnit, NUnit, and TestNG for Appium extent, the report becomes more helpful. It emphasizes more about the test report.

Advantages of Extent Report

• Step-by-step test execution information with pie charts and screenshots representation.
• Show the time taken for every test case execution.
• It can track multiple test cases within a single suite.
• Provide responsive UI.

Extent reports are written in HTML. Mainly the announcements contain two classes- ExtentReports class and ExtentTest class.

ExtentReports class: It generates HTML reports. This report will be created within your own folder means the path is user-specified. It can overwrite an existing HTML report. The default value becomes TRUE when a report is overwritten.

Syntax:

ExtentReports reports = new ExtentReports("Path of the directory to store the resultant HTML file", true/false);

ExtentTest class: This class log tests the previously generated HTML report.

Syntax:

ExtentTest test = reports.startTest("TestName");

Both these classes have some built-in methods. They are:

• startTest: Executes preconditions of a test case.
• endTest: Executes postconditions of a test case.
• Log: It’s a method. It logs each test step’s status onto the generated HTML report. It produces four value types- pass, fail, skip, and info, which we are elaborating on below.
• Flush: Creates an absolute new copy of the report and erases old data from the targeted report.

These values indicate the test status in the extent report-

• PASS
• FAIL
• SKIP
• INFO

Syntax

reports.endTest();
test.log(LogStatus.PASS,"Test Passed");
test.log(LogStatus.FAIL,"Test Failed");
test.log(LogStatus.SKIP,"Test Skipped");
test.log(LogStatus.INFO,"Test Info");

Generate extent report with NUnit

1. First you need to create an issuetrackerUTIL.java class. Thus we need to import the below packages. Then you will get an auto-generated Java document.

package <packagename>;

import io.restassured.RestAssured;
import io.restassured.builder.RequestSpecBuilder;
import io.restassured.http.ContentType;
import io.restassured.response.Response;
import io.restassured.specification.RequestSpecification;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.IInvokedMethod;
import org.testng.ITestResult;
import org.testng.SkipException;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.HashMap;

import static io.restassured.RestAssured.*;
import static io.restassured.path.json.JsonPath.from;
import org.apache.commons.lang3.StringUtils;

2. Create a listener to check the issue status.

package <packagenamehere>


import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;
import org.testng.SkipException;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

public class MethodIssueStatusListener implements IInvokedMethodListener {


@Override
public void afterInvocation(IInvokedMethod invokedMethod, ITestResult result) {
Issue issue = invokedMethod.getTestMethod()
.getConstructorOrMethod()
.getMethod()
.getAnnotation(Issue.class);

if (null != issue) {

IssueTrackerUtil.IssueStatus currentJiraStatus = IssueTrackerUtil.getStatus(issue.value());


if (IssueTrackerUtil.IssueStatus.OPEN==currentJiraStatus
|| IssueTrackerUtil.IssueStatus.IN_PROGRESS==currentJiraStatus
|| IssueTrackerUtil.IssueStatus.BLOCKED==currentJiraStatus
|| IssueTrackerUtil.IssueStatus.REOPENED==currentJiraStatus)
{
switch(result.getStatus()){
case ITestResult.FAILURE:
// no need to fail as we might have expected this already.
result.setStatus(ITestResult.SKIP);

if (issue.value()!= null) {
result.setAttribute( "issue", issue.value() );
}

break;

case ITestResult.SUCCESS:

}
}
}
}
@Override
public void beforeInvocation(IInvokedMethod invokedMethod, ITestResult result) {
// nothing to do

}

}

3. Create a @Issue interface to do the test with Jira

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Issue {
String value() default "";
}

4. After that, use the Listener

@Listeners({MethodIssueStatusListener.class})

Screenshot capture in the Extent report

Since Appium supports mobile app testing, the camera is an obvious feature to be tested. Almost every mobile application uses a camera, including taking pictures, screenshots, and other images oriented tasks. 

Here is a piece of code for taking screenshots. 

Also, some basic methods used in this code are described-

test.log(LogStatus.FAIL,test.addScreenCapture(capture(driver))+ "Test Failed");
public static String capture(WebDriver driver) throws IOException {
File scrFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
File Dest = new File("src/../BStackImages/" + System.currentTimeMillis()
+ ".png");
String errflpath = Dest.getAbsolutePath();
FileUtils.copyFile(scrFile, Dest);
return errflpath;
}

Methods:

addScreenCapture()

 This method is a member of the Extent Test class. It helps to get and add screenshots to the extent report. Thus, the log method uses this method to get a screenshot.

test.log(LogStatus.FAIL,test.addScreenCapture(capture(driver))+ "Test Failed");

getScreenShotAs()

This method captures screenshots of WebDriver instances. You must have to use this method when you’re taking screenshots of web drivers. It stores the screenshot in different output forms. 

Here, the output type is a FILE.

File scrFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);

 Then the output file will be stored in the’BStackImages’ folder in the ‘src’ folder. The  below statement creates the folder path-

File Dest = new File("src/../BStackImages/" + System.currentTimeMillis()+ ".png");

getAbsolutePath()

Use this method to place all error images in the destination folder.

Here is the complete statement for this-

 String errflpath = Dest.getAbsolutePath();
FileUtils.copyFile(scrFile, Dest);
return errflpath;

Now you have your extent report. Finally, you can get this report design from the dashboard

• Total number of test cases and associated test steps become visible. 
• You can see the start time, end time, and execution time of a test case. You can check it by clicking on a test case name.
• It provides the number of test cases passed, failed, and skipped.
• Display the percentage of the total number of test cases passed.
• The number of steps in each test case (both the passed and failed test cases).

Appium Bug report with JIRA

Several automation tools support JIRA integration to report a bug similar to a Selenium JIRA integration.

Here is a sample extent report by JIRA integration with the test Automation framework. 

The framework contains the below components-

• Appium
• Selenium
• Java 8 + TestNG
• Rest Assured

 Updating the bug reports in JIRA becomes easy when your bug is already fixed. 

Sample report by JIRA

compile group: 'com.aventstack', name: 'extentreports', version: '4.1.3'
use below extent-config.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<extentreports>
<configuration>
<!-- report theme -->
<!-- standard, dark -->
<theme>standard</theme>

<!-- document encoding -->
<!-- defaults to UTF-8 -->
<encoding>UTF-8</encoding>

<!-- protocol for script and stylesheets -->
<!-- defaults to https -->
<protocol>https</protocol>

<!-- title of the document -->
<documentTitle>ExtentReports 2.0</documentTitle>

<!-- report name - displayed at top-nav -->
<reportName></reportName>

<!-- report headline - displayed at top-nav, after reportHeadline -->
<reportHeadline>Automation Report</reportHeadline>

<!-- global date format override -->
<!-- defaults to yyyy-MM-dd -->
<dateFormat>yyyy-MM-dd</dateFormat>

<!-- global time format override -->
<!-- defaults to HH:mm:ss -->
<timeFormat>HH:mm:ss</timeFormat>


<enableCategoryView>true</enableCategoryView>
<enableAuthorView>true</enableAuthorView>
<enableExceptionView>true</enableExceptionView>
<enableTestRunnerLogsView>true</enableTestRunnerLogsView>

<!-- custom javascript -->
<scripts>
<![CDATA[
$(document).ready(function() {

});
]]>
</scripts>

<!-- custom styles -->
<styles>
<![CDATA[

]]>
</styles>
</configuration>
</extentreports>

App Automate for Bug reporting in Appium

You can opt for BrowserStack App Automate to report a bug for Appium testing. App Automate supports four types of integration: JIRA, Trello, GitHub, and Slack.

 Here is how the Jira integration works with App Automate-

Integration with Jira

• First, go to Report Bug>Integrate with Jira>Connect to my Jira Account to start the integration process. 
• For this purpose, you must provide your Jira account credentials, like hostname, email address, and Atlassian password.

Bug reporting with Jira

• Now, the JIRA integration with BrowserStack is completed. When you find a bug in your Appium test script, click on the ‘Report Bug’ button in the toolbox. This time you are filing a bug in Jira with this button.
• Then add annotation with the ‘Annotation Toolbar’ to the screenshot.
• After that, fill in all the details about the bug. Specify the project name, type of issue, and summary. Also, you add some description about the issue if you wish so.
• You can fill in additional fields with the ‘More Fields’ button. This is optional.
• Finally, click the ‘Create Issue’ button to report the issue.

Apart from this, also you can use the Jenkins integration by App Automate for Appium testing. Creating a report is simple with this plug-in. 

Steps are:

1. Go to Post-build Actions>Add post-build action.
2. Next select BrowserStack Test Report.
3. Finally, click on Save.

Now, you can see the result of the Appium test

Wrapping Up

• Additionally, it is quite straightforward to use Appium with BrowserStack and access a wide range of debugging tools, including text logs, network logs, visual logs, crash logs, and contextual log information.
• BrowserStack App Automate enables you to test native and hybrid mobile applications using the Appium automation framework on thousands of real Android and iOS devices. 
• Get on board with the BrowserStack infrastructure to run and debug Appium tests on a real device cloud. 

Debug in Appium with BrowserStack