Как установить allure windows 10

Введение в Allure и его возможности

Allure — это мощный инструмент для создания отчетов о тестировании, который помогает визуализировать результаты тестов в удобном и наглядном формате. Он поддерживает различные языки программирования и фреймворки, что делает его универсальным решением для большинства проектов. Allure позволяет легко отслеживать статус тестов, анализировать ошибки и улучшать качество кода.

Allure предоставляет следующие возможности:

• 📊 Визуализация результатов тестирования
• 📈 Поддержка различных языков программирования и фреймворков
• 🔄 Интеграция с CI/CD системами
• 📋 Поддержка различных типов отчетов (тесты, шаги, скриншоты, логи и т.д.)

Allure позволяет разработчикам и тестировщикам получать полную картину о состоянии тестов и качестве кода. Это особенно важно в больших проектах, где количество тестов может исчисляться сотнями или тысячами. Благодаря Allure, можно легко определить, какие тесты прошли успешно, какие провалились и почему. Это помогает быстро находить и исправлять ошибки, что в конечном итоге улучшает качество продукта.

Картинка с сайта: sky.pro

Установка и настройка Allure

Установка Allure

Для начала работы с Allure необходимо установить его на ваш компьютер. Существует несколько способов установки Allure, но мы рассмотрим самый простой — через Homebrew (для macOS) или Chocolatey (для Windows).

Установка на macOS

Для установки Allure на macOS воспользуйтесь Homebrew:

Homebrew — это популярный пакетный менеджер для macOS, который упрощает установку различных программ и утилит. Установка Allure через Homebrew занимает всего несколько минут и не требует сложных настроек.

Установка на Windows

Для установки Allure на Windows воспользуйтесь Chocolatey:

Chocolatey — это пакетный менеджер для Windows, который позволяет легко устанавливать и обновлять программы. Установка Allure через Chocolatey также проста и удобна.

Настройка Allure

После установки Allure необходимо настроить его для работы с вашим проектом. Обычно это включает в себя добавление зависимостей в проект и настройку конфигурационных файлов.

Настройка для Maven проекта

Добавьте следующие зависимости в ваш pom.xml:

Maven — это популярный инструмент для управления зависимостями и сборкой проектов на языке Java. Добавление зависимостей Allure в pom.xml позволяет интегрировать Allure с вашим проектом и использовать его возможности для создания отчетов о тестировании.

Настройка для Gradle проекта

Добавьте следующие зависимости в ваш build.gradle:

Gradle — это еще один популярный инструмент для управления зависимостями и сборкой проектов. Он используется в основном для проектов на языке Java и других JVM-языках. Настройка Allure в Gradle также проста и требует добавления всего одной строки в конфигурационный файл.

Создание тестов и интеграция с Allure

Создание тестов

Для демонстрации мы будем использовать JUnit 5. Создадим простой тестовый класс:

JUnit 5 — это популярный фреймворк для написания и выполнения тестов на языке Java. Он предоставляет удобный и понятный синтаксис для создания тестов, а также множество полезных аннотаций и утилит.

Интеграция с Allure

Для интеграции Allure с тестами необходимо использовать аннотации. Например, аннотация @Step позволяет описывать шаги теста, а аннотация @Attachment — добавлять вложения (скриншоты, логи и т.д.).

Аннотации Allure помогают структурировать тесты и делать их более понятными и читаемыми. Например, аннотация @Step позволяет описывать каждый шаг теста, что делает отчет более информативным. Аннотация @Attachment позволяет добавлять к отчету различные вложения, такие как скриншоты, логи и другие файлы.

Запуск тестов и генерация отчетов

Запуск тестов

Запустите тесты с помощью вашей системы сборки (Maven, Gradle и т.д.). Например, для Maven:

Запуск тестов с помощью Maven или Gradle позволяет автоматически выполнять все тесты в проекте и генерировать отчеты о результатах. Это удобно, так как не требует ручного запуска каждого теста и позволяет быстро получить информацию о состоянии проекта.

Генерация отчетов

После выполнения тестов необходимо сгенерировать отчет Allure. Для этого выполните следующую команду:

Эта команда запустит локальный сервер и откроет отчет в браузере. Отчет Allure предоставляет подробную информацию о каждом тесте, включая шаги, вложения и логи. Это позволяет легко находить и исправлять ошибки.

Анализ отчетов и лучшие практики

Анализ отчетов

Отчеты Allure предоставляют подробную информацию о каждом тесте, включая шаги, вложения и логи. Это позволяет легко находить и исправлять ошибки. Например, если тест провалился, отчет Allure покажет, на каком шаге произошла ошибка и какие данные были использованы. Это помогает быстро определить причину ошибки и исправить ее.

Лучшие практики

1. Используйте аннотации: Аннотации Allure помогают структурировать тесты и делать их более читаемыми. Например, аннотация @Step позволяет описывать каждый шаг теста, что делает отчет более информативным.
2. Добавляйте вложения: Скриншоты, логи и другие вложения помогают лучше понимать, что происходило во время выполнения теста. Например, если тест провалился, скриншот может показать, что именно пошло не так.
3. Интегрируйте с CI/CD: Интеграция Allure с системами CI/CD (например, Jenkins) позволяет автоматически генерировать отчеты после каждого запуска тестов. Это помогает поддерживать высокое качество кода и быстро находить и исправлять ошибки.
4. Регулярно обновляйте Allure: Новые версии Allure могут содержать важные улучшения и исправления ошибок. Регулярное обновление Allure помогает использовать все его возможности и получать более точные и информативные отчеты.
5. Используйте метки и категории: Метки и категории позволяют группировать тесты и упрощают их анализ. Например, вы можете создать метки для разных модулей вашего проекта и быстро находить тесты, относящиеся к конкретному модулю.
6. Пишите понятные и детализированные шаги: Чем более детализированными будут ваши шаги, тем легче будет анализировать отчеты и находить ошибки. Описывайте каждый шаг теста так, чтобы он был понятен даже тем, кто не знаком с вашим проектом.

Следуя этим рекомендациям, вы сможете эффективно использовать Allure для тестирования операций и улучшения качества вашего кода. Allure — это мощный инструмент, который помогает делать тестирование более прозрачным и понятным, а также ускоряет процесс нахождения и исправления ошибок.

Читайте также

Allure Framework is a flexible lightweight multi-language test report tool that not only shows a very concise representation of what have been tested in a neat web report form, but allows everyone participating in the development process to extract maximum of useful information from everyday execution of tests.

From the dev/qa perspective Allure reports shorten common defect lifecycle: test failures can be divided on bugs and broken tests, also logs, steps, fixtures, attachments, timings, history and integrations with TMS and bug-tracking systems can be configured, so the responsible developers and testers will have all information at hand.

From the managers perspective Allure provides a clear ‘big picture’ of what features have been covered, where defects are clustered, how the timeline of execution looks like and many other convenient things. Modularity and extensibility of Allure guarantees that you will always be able to fine-tune something to make Allure suit you better.

About ​

Copyright ​

The Allure reference guide is available as HTML documents. The latest copy is available at https://docs.qameta.io/allure/

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Get Help ​

There are several places to get help:

• Contact the community on Gitter. We also have a Russian-speaking room.

• Ask a question on Stack Overflow or Stack Overflow in Russian.

• Report bugs in GitHub issues.

How to Proceed ​

• Open the demo version to see what an Allure report looks like.

• Go to Get started to build a report for an existing project.

• Learn more about report structure and features.

• Integrate your favorite testing framework with Allure. Supported frameworks are grouped by language: Java, Python, JavaScript, Ruby, Groovy, PHP, .Net, and Scala.

Get Started ​

To generate your first report you will need to go through just a few simple steps:

• Downloading and installing Allure commandline application suitable for your environment.

• Locating test execution data that you have to build a report on.

Installing a commandline ​

Several options for Allure installation are currently supported:

Linux ​

For debian-based repositories a PPA is provided:

sudo apt-add-repository ppa:qameta/allure
sudo apt-get update 
sudo apt-get install allure

sudo apt-add-repository ppa:qameta/allure
sudo apt-get update 
sudo apt-get install allure

Mac OS X ​

For Mas OS, automated installation is available via Homebrew

brew install allure

brew install allure

Windows ​

For Windows, Allure is available from the Scoop commandline-installer.

To install Allure, download and install Scoop and then execute in the Powershell:

scoop install allure

scoop install allure

Also Scoop is capable of updating Allure distribution installations. To do so navigate to the Scoop installation directory and execute

\bin\checkver.ps1 allure -u

\bin\checkver.ps1 allure -u

This will check for newer versions of Allure, and update the manifest file. Then execute

scoop update allure

scoop update allure

to install a newer version. (documentation)

Manual installation ​

1. Download the latest version as zip archive from Maven Central.

2. Unpack the archive to allure-commandline directory.

3. Navigate to bin directory.

4. Use allure.bat for Windows or allure for other Unix platforms.

5. Add allure to system PATH.

Check the installation ​

Execute allure --version in console to make sure that allure is now available:

$ allure --version
2.0.1

$ allure --version
2.0.1

Test execution ​

Before building a report you need to run your tests to obtain some basic test report data. Typically it might be a junit-style xml report generated by nearly every popular test framework. For example, suppose you have test reports automatically created by surefire maven plugin stored in the target/surefire-reports:

Report generation ​

This is already enough to see the Allure report in one command:

allure serve /home/path/to/project/target/surefire-reports/

allure serve /home/path/to/project/target/surefire-reports/

Which generates a report in temporary folder from the data found in the provided path and then creates a local Jetty server instance, serves generated report and opens it in the default browser. It is possible to use a --profile option to enable some pre-configured allure setting. junit profile is enabled by default, you will learn more about profiles in the following section.

This would produce a report with a minimum of information extracted from the xml data that will lack nearly all of the advanced allure features but will allow you to get a nice visual representation of executed tests.

Картинка с сайта: huohuoren4.github.io

Report structure ​

Once you’ve got the idea what the report does look like. You will probably want to get more data-rich reports. You might have to consider using one of the Allure adaptors for your testing framework, which will allow to collect much more information. Jump to the integrations section to learn more about integration with testing frameworks.

Typical report consists of ‘Overview’ tab, navigation bar, several tabs for different kinds of test data representation and test case pages for each individual test. Each Allure report is backed by a tree-like data structure, that represents a test execution process. Different tabs allow to switch between the views of the original data structure thus giving a different perspective. Note that all tree-like representations including Behaviors, Categories, xUnit and Packages support filtering and are sortable.

Overview page ​

Entry point for every report would be the ‘Overview’ page with dashboards and widgets:

Картинка с сайта: huohuoren4.github.io

Overview page hosts several default widgets representing basic characteristics of your project and test environment.

• Statistics — overall report statistics.

• Launches — if this report represents several test launches, statistics per launch will be shown here.

• Behaviors — information on results aggregated according to stories and features.

• Executors — information on test executors that were used to run the tests.

• History Trend — if tests accumulated some historical data, it’s trend will be calculated and shown on the graph.

• Environment — information on test environment (see how to define environment).

Home page widgets are draggable and configurable. Also, Allure supports it’s own plugin system, so quite different widget layouts are possible.

Navigation bar is collapsible and enables you to switch into several of the basic results overview modes.

Categories ​

Categories tab gives you the way to create custom defects classification to apply for test results.

Картинка с сайта: huohuoren4.github.io

Suites ​

On the Suites tab a standard structural representation of executed tests, grouped by suites and classes can be found.

Картинка с сайта: huohuoren4.github.io

Graphs ​

Graphs allow you to see different statistics collected from the test data: statuses breakdown or severity and duration diagrams.

Картинка с сайта: huohuoren4.github.io

Timeline ​

Timeline tab visualizes retrospective of tests execution, allure adaptors collect precise timings of tests, and here on this tab they are arranged accordingly to their sequential or parallel timing structure.

Картинка с сайта: huohuoren4.github.io

Behaviors ​

For Behavior-driven approach, this tab groups test results according to Epic, Feature and Story tags.

Картинка с сайта: huohuoren4.github.io

Packages ​

Packages tab represents a tree-like layout of test results, grouped by different packages.

Картинка с сайта: huohuoren4.github.io

Test case page ​

From some of the results overview pages described above you can go to the test case page after clicking on the individual tests. This page will typically contain a lot of individual data related to the test case: steps executed during the test, timings, attachments, test categorization labels, descriptions and links.

Картинка с сайта: huohuoren4.github.io

Features ​

This section describes the main features of Allure. For example, you can group your tests by stories or features, attach files, and distribute assertions over a set of custom steps, among other features. All features are supported by Java test frameworks, so we only provide Java examples here. For details on how a particular adapter works with the test framework of your choice, refer to the adapter guide.

Flaky tests ​

In real life not all of your tests are stable and always green or always red. A test might start to «blink» i.e. it fails from time-to-time without any obvious reason. You could disable such a test, that is a trivial solution. However what if you do not want to do that? Say you would like to get more details on possible reasons or the test is so critical that even being flaky it provides helpful information? You have an option now to mark such tests in a special way, so the resulting report will clearly show them as unstable:

@Flaky
public void aTestWhichFailsFromTimeToTime {
     ...
}

@Flaky
public void aTestWhichFailsFromTimeToTime {
     ...
}

Here is what you get in the report if such a test failed:

Environment ​

To add information to Environment widget just create environment.properties (or environment.xml) file to allure-results directory before report generation.

# environment.properties
Browser=Chrome
Browser.Version=63.0
Stand=Production

# environment.properties
Browser=Chrome
Browser.Version=63.0
Stand=Production

<!-- environment.xml -->
<environment>
    <parameter>
        <key>Browser</key>
        <value>Chrome</value>
    </parameter>
    <parameter>
        <key>Browser.Version</key>
        <value>63.0</value>
    </parameter>
    <parameter>
        <key>Stand</key>
        <value>Production</value>
    </parameter>
</environment>

<!-- environment.xml -->
<environment>
    <parameter>
        <key>Browser</key>
        <value>Chrome</value>
    </parameter>
    <parameter>
        <key>Browser.Version</key>
        <value>63.0</value>
    </parameter>
    <parameter>
        <key>Stand</key>
        <value>Production</value>
    </parameter>
</environment>

Categories ​

There are two categories of defects by default:

• Product defects (failed tests)

• Test defects (broken tests)

To create custom defects classification add categories.json file to allure-results directory before report generation.

// categories.json
[
  {
    "name": "Ignored tests",   // (mandatory) category name
    "matchedStatuses": ["skipped"]  // (optional) list of suitable test statuses. Default ["failed", "broken", "passed", "skipped", "unknown"]
  },
  {
    "name": "Infrastructure problems",
    "matchedStatuses": ["broken", "failed"],
    "messageRegex": ".*bye-bye.*"  // (optional) regex pattern to check test error message. Default ".*"
  },
  {
    "name": "Outdated tests",
    "matchedStatuses": ["broken"],
    "traceRegex": ".*FileNotFoundException.*"  // (optional) regex pattern to check stack trace. Default ".*"
  },
  {
    "name": "Product defects",
    "matchedStatuses": ["failed"]
  },
  {
    "name": "Test defects",
    "matchedStatuses": ["broken"]
  }
]

// categories.json
[
  {
    "name": "Ignored tests",   // (mandatory) category name
    "matchedStatuses": ["skipped"]  // (optional) list of suitable test statuses. Default ["failed", "broken", "passed", "skipped", "unknown"]
  },
  {
    "name": "Infrastructure problems",
    "matchedStatuses": ["broken", "failed"],
    "messageRegex": ".*bye-bye.*"  // (optional) regex pattern to check test error message. Default ".*"
  },
  {
    "name": "Outdated tests",
    "matchedStatuses": ["broken"],
    "traceRegex": ".*FileNotFoundException.*"  // (optional) regex pattern to check stack trace. Default ".*"
  },
  {
    "name": "Product defects",
    "matchedStatuses": ["failed"]
  },
  {
    "name": "Test defects",
    "matchedStatuses": ["broken"]
  }
]

Test result falls into the category if its status is in the list and both error message and stack trace match the pattern.

Allure Report is an open-source multi-language test reporting tool. It builds a detailed representation of what has been tested and extracts as much information as possible from everyday test execution.

In this guide, we’ll take a journey through the main steps toward creating your first Allure report and discover all the fancy features it brings to routine automated testing reports.

Since Allure Report has various integrations with various testing frameworks on different programming languages, there is a chance that some steps will vary for each reader, so feel free to jump into the official documentation page for details.

Installing Allure Report

As always, the first step is to install the Allure library. The exact steps vary depending on your OS:

Homebrew (for macOS and Linux)

For Linux and macOS, automated installation is available via Homebrew

brew install allure

Scoop (for Windows)

For Windows, Allure is available from the Scoop command-line installer.

To install Allure, download and install Scoop, and then execute the following command in Powershell:

scoop install allure

System package manager (for Linux)

1. Go to the latest Allure Report release on GitHub and download the allure-*.deb or allure-*.rpm package, depending on which package format your Linux distribution supports.

2. Go to the directory with the package in a terminal and install it.

For the DEB package:

sudo dpkg -i allure_2.24.0-1_all.deb

For the RPM package:

sudo rpm -i allure_2.24.0-1.noarch.rpm

NPM (any system)

1. Make sure Nodejs and NPM are installed.

2. Make sure Java version 8 or above is installed and its directory is specified in the JAVA_HOME environment variable.

3. In a terminal, go to the project’s root directory for which you want to use Allure Report. Run this command:

npm install --save-dev allure-commandline

This installation method only makes Allure Report available in the given project’s directory. Also note that the commands for running Allure Report must be prefixed with npx, for example:

npx allure-commandline serve

From an archive (any system)

1. Make sure Java version 8 or above is installed and its directory is specified in the JAVA_HOME environment variable.

2. Go to the latest Allure Report release on GitHub and download the allure-*.zip or allure-*.tgz archive.

3. Uncompress the archive into any directory. The Allure Report can now be run using the bin/allure or bin/allure.bat script, depending on the operating system.

With this installation method, the commands for running Allure Report must contain the full path to the scripts, for example:

D:\Tools\allure-2.24.0\bin\allure.bat serve

Check the installation

Once you’ve followed one of the above steps, it’s a good idea to check if Allure is now available on your system:

$ allure --version
2.24.0

Plugging Allure Report into code

The next step is to provide all the necessary dependencies in a configuration file so that your build tool can use Allure.

Each framework and build tool has its own configuration settings, so the best way to get a clue on adding dependencies is to look for an example at the documentation page or get an example at GitHub.

Adding annotations to the report

After plugging Allure Report into the codebase, we can run it. You will get a pretty report, though it won’t have much to tell (at least compared to its full potential):

Картинка с сайта: qameta.io

We need to annotate the tests to provide all the necessary data to Allure. There are several types of annotations:

• Descriptive annotations that provide as much information as possible about the test case and its context
• The step annotation, the one that allows Allure to build nice and detailed test scenarios
• Parameterized annotations that can accept values from the test’s own inputs

Descriptive annotations

Let’s go over the existing annotations one by one.

@Epic, @Feature, @Story:

This is a set of annotations designed to make test-case tree grouping more flexible and informative. The annotations follow the Agile approach for task definition. These annotations may be implemented on the class or on the method level.

Epic defines the highest-level task that will be decomposed into features. Features will group specific stories, providing an easily readable structure.

As a story is the lowest part of the epic-feature-story hierarchy, the class-level story adds data to all class methods.

@Description

An annotation that provides a detailed description of a test method/class to be displayed in the report.

@Owner

A simple annotation to highlight the person behind the specific test case so that everyone knows whom to ask for a fix in case of a broken/failed test. Quite useful for large teams.

@Severity

In Allure, any @Test can be defined with a @Severity annotation that accepts any of the following values:

• SeverityLevel.BLOCKER
• SeverityLevel.CRITICAL
• SeverityLevel.NORMAL
• SeverityLevel.MINOR
• SeverityLevel.TRIVIAL

The severity level will be displayed in the report so that the tester understands how serious the problem is if a test has failed.

Sample Tests

Let’s take a look at an example of these annotations in Java (annotations for any other language will look similar):

public class AllureExampleTest {

    @Test
    @Epic("Sign In flow")
    @Feature("Login form")
    @Story("User enters the wrong password")
    @Owner("Nicola Tesla")
    @Severity(SeverityLevel.BLOCKER)
    @Description("Test that verifies a user cannot enter the page without logging in")
    public void annotationDescriptionTest() {}

    /**
     * JavaDoc description
     */
    @Test
    @Description(useJavaDoc = true)
    public void javadocDescriptionTest() {}
}

The Step annotation

Detailed reporting with steps is one of the features people love about Allure Report. The @Step annotation makes this feature possible by providing a human-readable description of any action within a test. Steps can be used in various testing scenarios. They can be parametrized, make checks, have nested steps, and create attachments. Each step has a name.

To define steps in code, each method should have a @Step annotation with a String description; otherwise, the step name equals the annotated method name.

A step can extract the names of fields using reflection so that they can be used to, e.g., provide the name of the step.

Here are several examples of what the Step annotation looks like in code:

package io.qameta.allure.examples.junit5;

import io.qameta.allure.Allure;
import io.qameta.allure.Step;
import org.junit.jupiter.api.Test;

public class AllureStepTest {

    private static final String GLOBAL_PARAMETER = "global value";

    // A test inside which a @Step-annotated method is used
    @Test
    public void annotatedStepTest() {
        annotatedStep("local value");
    }

    // A test with a step implemented using a lambda
    @Test
    public void lambdaStepTest() {
        final String localParameter = "parameter value";
        Allure.step(String.format("Parent lambda step with parameter [%s]", localParameter), (step) -> {
            step.parameter("parameter", localParameter);
            Allure.step(String.format("Nested lambda step with global parameter [%s]", GLOBAL_PARAMETER));
        });
    }

    // The methods that can be used as steps
    @Step("Parent annotated step with parameter [{parameter}]")
    public void annotatedStep(final String parameter) {
        nestedAnnotatedStep();
    }

    @Step("Nested annotated step with a global parameter [{this.GLOBAL_PARAMETER}]")
    public void nestedAnnotatedStep() {

    }

Parameterized annotations

@Attachment

This annotation allows attaching a String or Byte array to the report. It is very helpful if you need to show a screenshot or a failure stack trace in your test results.

@Link

It’s just as the name suggests: if you need to add a link to the test, be it a reference or a hyperlink, this is the annotation you need.

It takes several parameters:

• name: link text
• url: an actual link
• type: type of link
• value: similar to name

@Muted

An annotation that excludes a test from a report.

@TmsLink

A way to link a result with a TMS object, if you use any. Allows entering just the test case ID that will be added to the pre-configured (via allure.link.tms.pattern) URL. The annotation takes a String value, the link to the management system. For example, if the link to our test case on the TMS is https://tms.yourcompany.com/browse/tc-12, then we can use tc-12 as the value.

Running Allure Report

Local launch

Running Allure Report locally is a great way to get started with it. However, remember that local execution does not provide execution, result history, or trend graphs.

Generally speaking, the easiest way to try Allure Report is to download a pre-made empty project from Allure Start. There, you can select any tech stack you want; download the project, add some sample tests, and build it.

However, here, we’re going to go with an example that already has some tests in it, because we want to show you Report’s features. You can follow along by downloading the code from the GitHub link. The example uses JUnit 5 and Gradle.

Once you’ve downloaded the project and built it with Gradle, you can run the tests with the./gradlew test command. As soon as they have been executed, Gradle will store the test results in the target directory.

Let’s take the data and build a report! With the allure serve /path/to/allure-results command, we start an Allure Report instance (the allure-results folder is usually in the build folder in the root of your project). It builds a local web report which automatically opens as a page:

Картинка с сайта: qameta.io

CI (Jenkins, TeamCity, and Bamboo)

Instead of running Report locally, you can generate on a CI server. Allure Report has great integrations with multiple CI systems. Each system setup has specificities, and we won’t cover them all in this post; you can follow the Documentation page for steps to create a report with a CI system (e. g. Jenkins).

Allure Report Features

Now that we’ve set up the basic functionality, you can build upon it with other features of Allure Report.

Attachments

Often, it’s not enough to read the list of executed steps; you need to closely examine the system under test. Allure Report allows you to automatically gather all kinds of data about the system you’re testing — take screenshots, gather webpage source code, etc. For this, we either leverage the existing functionality of your framework or create this functionality from scratch. Once collected, the data is attached to your report:

Картинка с сайта: qameta.io

This way, you’ve got exhaustive information necessary to diagnose and reproduce bugs.

Integrations

Allure Report is polyglot: it works with pretty much any popular framework. The architecture of Report has been designed to simplify the process of making integrations, and we’ve put a lot of thought and effort into creating them.

A full list of integrations is available on our website. Detailed instructions for different frameworks are available in our documentation (for Java, Python, etc.).

The integrations, together with the steps, help hide any technical details of your tests (such as programming language and test framework), so the interface presents the test scenario in a form familiar to manual testers or managers.

Categories

Categories are one of the most time-saving features of Allure Report. They provide simple automation for fail resolution. There are two categories of defects by default:

• Product defects (failed tests)
• Test defects (broken tests)

Categories are fully customizable via simple JSON configuration. To create custom defects classification, add a categories.json file to the allure-results directory before report generation.

• Open JSON template [ {"name": "Ignored tests", "matchedStatuses": ["skipped"] }, {"name": "Infrastructure problems", "matchedStatuses": ["broken", "failed"], "messageRegex": ".*bye-bye.*"}, {"name": "Outdated tests", "matchedStatuses": ["broken"], "traceRegex": ".*FileNotFoundException.*"}, {"name": "Product defects", "matchedStatuses": ["failed"] }, {"name": "Test defects", "matchedStatuses": ["broken"] } ]

The JSON includes the following data:

• (mandatory) Category name
• (optional) list of suitable test statuses. The default ones are: [«failed», «broken», «passed», «skipped», «unknown» ]
• (optional) regex pattern to check the test error message. Default value: «._»
• (optional) regex pattern to check the stack trace. Default value: «.»

A test result falls into a category if its status is in the list and both the error message and the stack trace match the pattern.

If you’re using allure-maven  or allure-gradle plugins, categories.json  file can be stored in the test resources directory.

Parameterized tests

Allure knows how to work with parameterized automated tests. Let’s take a JUnit test as an example. First, let’s create a test class with a parameterized test:

@Layer("rest")
@Owner("baev")
@Feature("Issues")
public class IssuesRestTest {

    private static final String OWNER = "allure-framework";
    private static final String REPO = "allure2";

    private final RestSteps steps = new RestSteps();

    @TM4J("AE-T1")
    @Story("Create new issue")
    @Microservice("Billing")
    @Tags({@Tag("api"), @Tag("smoke")})
    // The important annotation:
    @ParameterizedTest(name = "Create issue via api")
    @ValueSource(strings = {"First Note", "Second Note"})
    public void shouldCreateUserNote(String title) {
        steps.createIssueWithTitle(OWNER, REPO, title);
        steps.shouldSeeIssueWithTitle(OWNER, REPO, title);
    }

After execution, Allure provides the parameterized test run results as a set of tests, with the value of the parameter specified in the overview of each test:

Картинка с сайта: qameta.io

If any tests fail, Allure provides detailed information about that particular case.

Retries

Retires are executions of the same test cases (a signature is calculated based on the test method name and parameters) within one test suite execution, e.g., when using TestNG IRetryAnalyzer or JUnit retry Rules. Unfortunately, this is not supported for local runs.

Test History

Allure Report supports history for tests. At each report generation during the build, the Allure Plugin for Jenkins will try to access the working directory of the previous build and copy the contents of the allure-report/history folder to the current report.

Currently, the history entry for the test case stores information for up to 5 previous results, and it is not available for local runs.

Report Structure and Dashboards

Now, let’s go over the structure of a report, as it is presented in the main menu:

Overview

Картинка с сайта: qameta.io

The default page would be the ‘Overview’ page with dashboards and widgets. The page has several default widgets representing the essential characteristics of your project and test environments:

• Statistics — overall report statistics.
• Launches — statistics by launch, provided that the report is based on multiple launches.
• Behaviors — information on results aggregated according to stories and features.
• Executors — information on test executors used to run the tests.
• History Trend — if tests accumulate some historical data, a trend will be calculated and shown on the graph.
• Environment — information on the test environment.

Home page widgets are draggable and configurable. Also, Allure supports its own plugin system, so you can have very different widget layouts.

Categories

Картинка с сайта: qameta.io

This page shows all defects. Assertion failures are reported as ‘Product defects’, whereas failures caused by exceptions are shown in the report as ‘Test defects’.

Suites, Behaviors, and Packages

Картинка с сайта: qameta.io

Three tabs that show the test case tree with tests grouped by:

• Suites. This tab displays test cases based on the suite executed.
• Behaviors. Here, the tree is based on stories and feature annotations.
• Packages. In this tab, the test cases are grouped by package names.

Graphs

Картинка с сайта: qameta.io

On this tab, test results are visualized with charts. The default configurations provide:

• A pie chart with general execution results.
• A duration trend of test case execution. This is a nice feature if you need to investigate which tests require more time and optimization.
• The retries trend shows how tests are re-executed during a single test run.
• The categories trend shows the categories of defects encountered.

Timeline

Картинка с сайта: qameta.io

This view displays the timeline of executed tests.

Conclusion

Allure Report is being used by millions of people, and it has become a mainstay of test automation, which is why we will continue supporting and perfecting this tool. If you’re spending too much time digging through your test results, or if you want to show them to other people who don’t code — Allure Report is the tool for you.