How to generate Allure Reports in Selenium and TestNG?

The first time I used Selenium, I was happy just seeing tests pass or fail in the console. That worked well enough until the test suite grew. Once dozens or even hundreds of tests were running in every build, scrolling through logs to understand what failed became frustrating and time-consuming.

That’s where Allure Reports started making a real difference for me. Instead of digging through raw test output, I could see failed tests, screenshots, stack traces, execution history, and other useful details in one interactive report. It made debugging faster and gave everyone on the team, not just the person who wrote the test, a much clearer picture of what happened.

Here you’ll learn how to set up Allure Reports with Selenium, generate your first report, customize it with annotations and attachments, and integrate it into your automation workflow.

What is Allure Reporting?

Allure Reporting is an open-source reporting tool that integrates with Selenium and TestNG. It collects raw test results and generates reports that present information clearly and interactively.

The reports indicate which tests passed, failed, or were skipped. They include detailed information such as step-by-step execution logs, screenshots, and error messages. This level of detail makes debugging easier.

What Makes It Worth Using

TestNG already tells you whether a test passed or failed, but that’s only part of the story. When a test fails, you still need to understand what happened, where it failed, and whether it’s part of a larger pattern.

• Understand failures faster: Instead of reading long console logs, you can open a failed test and immediately view the execution steps, stack trace, screenshots, and other supporting evidence in one place.
• See how a test reached the failure: By adding step annotations, you can follow the entire execution flow and quickly identify the action that caused the test to fail.
• Keep all debugging evidence together: Attach screenshots, browser logs, videos, API responses, or other files directly to the test result, so you don’t have to collect information from multiple tools.
• Track quality over time: Allure preserves execution history, making it easier to spot flaky tests, recurring failures, or improvements across multiple test runs.
• Share reports with the whole team: Interactive HTML reports are much easier for developers, QA engineers, and stakeholders to review than raw TestNG output, especially when reports are automatically published through your CI/CD pipeline.

Setting Up Allure with Selenium and TestNG

Getting started with Allure doesn’t take much. If you already have a Selenium project using TestNG, you’ll mainly be adding the Allure adapter, configuring Maven to generate reports, and enriching your tests with annotations.

Before you begin, make sure you have:

• Java and Maven installed.
• A working Selenium project that uses TestNG.
• Maven as your build tool (the examples below use Maven).

Add the Allure TestNG Adapter

The first step is to include the Allure TestNG dependency in your pom.xml.

<dependency>

    <groupId>io.qameta.allure</groupId>

    <artifactId>allure-testng</artifactId>

    <version>2.13.9</version>

</dependency>

This adapter listens to your TestNG test execution and generates the result files that Allure later converts into an interactive HTML report.

Configure Maven to Generate Reports

Next, add the Allure Maven plugin to your project.

<build>

    <plugins>

        <plugin>

            <groupId>io.qameta.allure</groupId>

            <artifactId>allure-maven</artifactId>

            <version>2.10.0</version>

            <executions>

                <execution>

                    <id>allure-report</id>

                    <phase>verify</phase>

                    <goals>

                        <goal>report</goal>

                    </goals>

                </execution>

            </executions>

        </plugin>

    </plugins>

</build>

With this configuration in place, Maven generates an Allure report after your tests finish executing, so you don’t have to create reports manually every time you run the suite.

Add Context to Your Test Results

At this point, Allure is ready to collect test results. To make those reports genuinely useful, start adding Allure annotations such as @Description, @Step, and @Severity to your tests. Let’s work more on this in this next section.

import io.qameta.allure.*;

import org.testng.Assert;

import org.testng.annotations.Test;

public class SampleTest {

    @Test

    @Description("Verify the homepage title")

    public void testHomePageTitle() {

        Allure.step("Navigate to homepage");

        driver.get("https://example.com");

        Allure.step("Validate page title");

        String title = driver.getTitle();

        Assert.assertEquals(title, "Example Domain");

    }

}

When you open the generated report, you’ll see the test description followed by each execution step in the order it ran. If the test fails, you’ll know exactly which action caused the failure instead of relying only on a stack trace.

As your test suite grows, this additional context makes debugging much faster for both the person who wrote the test and anyone else reviewing the report.

Running Tests and Generating Allure Reports

Once tests are annotated and dependencies added, run your tests normally. Test results are stored in the allure-results folder.

To generate the interactive report, you need the Allure command-line tool. Install it using:

• Homebrew on macOS:

brew install allure

• Generate the report:

allure generate allure-results --clean -o allure-report

• Open the report in your browser:

allure open allure-report

The report shows an interactive dashboard with all tests, their statuses, and detailed logs.

Understanding Each Component

The first time you open an Allure Report, you’ll notice that it contains much more than a pass or fail summary. Each section helps answer a different question, whether you’re trying to understand the overall health of a test run or investigate why a specific test failed.

Overview Dashboard

This is usually where I start. It gives you a quick snapshot of the entire test run, including how many tests passed, failed, were skipped, or retried. If something looks unusual, you know immediately whether it’s an isolated failure or a wider problem affecting the suite.

Test Suites and Test Cases

Once I know which tests failed, I jump into the test suites. They’re organised by your project structure, making it easy to navigate from a package or class down to an individual test without scrolling through hundreds of results.

Steps and Attachments

This is probably the section I spend the most time in. Every recorded step appears in the order it was executed, and if you’ve attached screenshots, logs, or other debugging artifacts, they’re available alongside the failure. It saves a lot of time compared to piecing information together from different tools.

Graphs and Trends

A single failed build doesn’t always tell the whole story. The trends view helps you see whether failures are becoming more frequent, whether a feature has stabilised, or if the same flaky tests keep reappearing across multiple runs.

Behaviours

If you’ve organised your tests using Allure annotations, this view groups them by features, epics, or user stories instead of technical classes. I’ve found this especially useful when discussing test coverage with product managers or other stakeholders who don’t think in terms of test classes.

Categories

Not every failure deserves the same attention. Categories group similar failures together, making it easier to distinguish between assertion failures, environment issues, setup problems, or other recurring error types. That helps you prioritise fixes instead of investigating every failed test individually.

Retries

Retries are helpful for spotting flaky tests. Rather than only showing the final result, Allure records every retry attempt, making it easier to tell whether a test genuinely recovered or simply passed on a second attempt. Over time, this helps you identify unstable tests that need attention instead of repeatedly relying on retries.

Common Challenges with Allure

While Allure is powerful, it has constraints that can affect complex test automation projects:

• No automatic flaky test detection: Allure shows retries but does not flag flaky tests explicitly. Developers must manually analyze retry patterns to identify instability.

• Limited failure categorization: Error grouping and prioritization require manual categorization or external tools. This slows down triage for large test suites.
• No built-in support for test reruns from reports: Developers cannot directly trigger failed test reruns via Allure UI, slowing down debugging cycles.
• Fragmented log consolidation: Logs and environment data come from separate sources. Allure does not consolidate these for unified failure analysis.
• No built-in test health metrics or quality gates: Teams must build custom dashboards or CI steps to enforce minimum quality standards based on test results.
• Reports require manual setup and maintenance: Configuring Allure with Selenium and TestNG, especially in CI environments, requires ongoing effort and troubleshooting.

Where Does BrowserStack Step In?

While Allure provides a solid foundation for visualizing test executions and attaching artifacts, it often requires significant manual effort to effectively identify flaky tests, group failures, and correlate logs across multiple test environments.

BrowserStack fills these gaps by offering an end-to-end test reporting, failure RCA, and analytics solution. It combines automated stability analysis, intelligent failure grouping, and seamless rerun capabilities directly within its dashboard.

Here are some ways BrowserStack Test Reporting and Analytics improve test reporting:

• Real Device Cloud: Tests run on actual mobile and desktop devices hosted by BrowserStack. Combined with advanced reporting, this exposes platform-specific issues that Allure alone might miss.
• Automatic flaky test detection: BrowserStack tracks test stability trends and automatically flags flaky tests. Alerts notify developers immediately, reducing wasted debugging time.
• AI-powered error categorization: Failures are grouped by root cause using machine learning. This prioritizes bugs that impact the most users or block release pipelines.
• Built-in test rerun capabilities: Developers can rerun failed tests directly from the BrowserStack dashboard on real devices or browsers, speeding up verification without leaving the tool.
• Detailed test analytics: Get insights into quality metrics like failure rate, performance, and top unique errors.
• Timeline debugging with event sequencing: The dashboard shows step-by-step test execution timelines across browsers and devices, allowing teams to track exactly when failures happen.
• Quality gates: Teams configure widgets and build quality gates to monitor test health continuously. Failed gates can block merges, improving release confidence.

Talk to an Expert

Getting More Value from Your Reports

Setting up Allure is only the first step. The quality of your reports depends on how much context you include and how consistently your team uses it. Over time, I’ve found these practices make the biggest difference.

• Write reports for the next person reading them: Add meaningful descriptions and step names instead of generic labels like Step 1 or Validate. When someone opens a failed test weeks later, they should understand what the test was trying to do without reading the source code.
• Attach evidence that helps debugging: Screenshots, browser logs, API responses, or videos can save a lot of investigation time. I usually attach them only when a test fails, otherwise reports become unnecessarily large and harder to navigate.
• Break complex tests into smaller steps: Long tests are much easier to follow when each major action is recorded separately. If a failure occurs, you can immediately see which part of the workflow caused it instead of tracing the entire execution.
• Generate reports automatically: Allure becomes far more useful when every test run produces a report without any manual effort. Integrating it into your CI/CD pipeline gives the team a consistent place to review results after every build.
• Keep an eye on flaky tests: A passing retry shouldn’t always be treated as a success. If the same tests keep failing intermittently, use the execution history to identify patterns and fix the underlying instability.
• Organise tests consistently: Labels such as features, epics, or modules make reports much easier to navigate, especially as your automation suite grows. A little organisation early on saves a lot of searching later.
• Keep reports focused: It’s tempting to attach every screenshot and log file, but too much information can make reports slow to load and difficult to scan. Include the evidence that’s useful for debugging and avoid everything else.

Conclusion

Once you’re running hundreds of tests across multiple builds, interactive reports with execution steps, screenshots and logs make it much easier to understand failures and collaborate with the rest of the team.

The best results come from treating Allure as more than just a reporting tool. Add meaningful annotations, attach useful debugging evidence, and generate reports automatically as part of your CI/CD pipeline.

Combined with Selenium and TestNG, Allure gives you a clearer view of your test health and helps you spend less time searching through logs and more time fixing real issues.

Try BrowserStack for Free