logo
Solutionunittest

JUnit 5 tags and filters

Views: 1231 Created: 2019-06-12 Read time: 5 minutes
Post Preview
Tags:

1)     Preface

Sometimes or actually most of the time, we would like to have control over our test suite content during build time. We would like to include only certain types of tests, for example, and exclude the rest. Depending on the environment we are building our application on, the set of included tests might be different. Development, staging, user acceptance, pre-production, release; all these environments might not be prepared or have a need to run all the possible tests we have. This is why Junit 5 is equipped with tagging and filtering support, which gives us a lot of flexibility during the building phase.

 

2)     Junit 5 @Tag: introduce grouping into your tests

The first step in preparing your flexible build is to place appropriate @Tag annotations on test methods or test classes:

 

Test Code


<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M3</version>
            <configuration>
                <groups>crucial</groups>
                <excludedGroups>legacy, auxiliary</excludedGroups>
            </configuration>
        </plugin>
    </plugins>
</build>

 

As we can see in the example above, tags can be stacked and combined, making it possible for a test method to have multiple ones.

 

3)     Filtering tests with Maven

Once our test methods are tagged, we are ready to get them into action by using filtering. Let us first take a look at how to filter out specific unit tests by configuring the Maven Surefire plugin:

 


<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M3</version>
            <configuration>
                <groups>crucial</groups>
                <excludedGroups>legacy, auxiliary</excludedGroups>
            </configuration>
        </plugin>
    </plugins>
</build>

 

Tip
Tip

 

When we want to configure our integration testing suite, we set-up the Maven Failsafe plugin accordingly:

 


<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>3.0.0-M3</version>
            <configuration>
                <groups>api, data-access</groups>
                <excludedGroups>regression</excludedGroups>
            </configuration>
        </plugin>
    </plugins>
</build>

 

Tip
Tip

 

Note
Note

 

4)     Filtering tests with Gradle

Gradle makes it very simple for us to filter out exactly what we need during the build phase:

 


test {
    useJUnitPlatform {
        includeTags 'slice'
        excludeTags 'security', 'regression', 'load'
    }
}

 

Note
Note

 

5)     Tagging and filtering real world example: preface

To be honest, I do not think that tagging and filtering are of any use regarding the unit tests. We should always strive to include all the unit tests in every single build. That is raging from development to release environment. With the parallel and concurrent strategies, we can bring down even the most extensive unit testing suites execution time to a matter of a few seconds. If written per conventions, then none of these tests should rely on any environment and its configuration. That means it is always safe to run them.

 

Now the real benefit of tagging and filtering is in the IT world. In chess, the beginning of the game is driven by openings. They are a set of widely agreed best initial moves. Based on that the first part of the game is trivial and predictable. The real war and strategies come into play in the middle game. This is where the actual level of each of the players is disclosed. Nothing is standard here, and the player has to adapt accordingly for each of the games which are unique from that point on.

Similarly, in the automated testing world, the way we approach unit testing is kind of similar in every project. Just like with the openings in chess, we simply follow the agreed-upon best paths. When we get to the IT stage, we have to step back a little, assess the situation and come up with a unique strategy. A strategy that is tailored for our particular application.

 

6)     Junit 5 Tagging: real world example

As per the previous section, there are many ways to tackle IT's, but let's try to define a set of types of these tests:

 

Image!it-types!Image

 

Let's try to go through each of these categories in more detail:

       Functional Test: anything that tries to verify the actual business logic contained in our source code.

       Slice Test: we select a set of components and verify the behaviour inside. This is always narrower than an E2E test ranging from the API all the way down the access layer. It is also usually broader than, for example, API exclusive IT's.

       E2E: broadest possible tests that range from the outer API to the data access layer. These cover the most logic possible but are hardest to maintain and write.

       API Test: a group of tests that exercise the accessibility of our application exclusively.

       Data Access Test: a group of tests that exercise the way the data is fetched and persisted exclusively.

       Security Test: we focus on authentication and authorisation features here only. It is essential to cover all the possibilities here. It only takes one mistake to let someone unauthorised into the system.

       Performance Test: this includes load and stress testing. In java, for example, this is done by the use of tools like JMeter. These tests are usually meant to run on a particular and dedicated environment that is isolated and stable.

 

The next step would be creating custom meta-annotations that combine and group these tests in a more manageable way, for example:

 

Slice Test:


@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
@Tag("functional")
@Tag("slice")
@interface SliceTest {
}

 

API Test:


@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
@Tag("functional")
@Tag("accessibility")
@Tag("api")
@interface ApiTest {
}

 

Stress:


@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
@Tag("non-functional")
@Tag("performance")
@Tag("stress")
@interface StressTest {
}

 

Having all of the groups covered and appropriate test methods annotated we are ready to configure our builds as we please.

 

Tip
Tip

 

7)     Junit 5 filtering: real world example

Let us try to use the Maven Surefire plugin to specify which IT's should be included in different environments:

 

Development Environment:


<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>3.0.0-M3</version>
	<configuration>
		<groups>slice</groups>
	</configuration>
</plugin>

In this case, we want to the bare minimum when it comes to the invocation of tests in our builds. Speed is the key here. We want to make sure that all unit tests pass along with the crucial IT's.

 

Staging Environment:


<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>3.0.0-M3</version>
	<configuration>
		<groups>functional, stress</groups>
	</configuration>
</plugin>

Here it is all about the performance. First, we make sure the system works as expected by invoking all the functional tests. Then we bombard it with as many stress tests as possible.

 

Release Environment:


<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>3.0.0-M3</version>
	<configuration>
		<excludedGroups>performance</excludedGroups>
	</configuration>
</plugin>

Just before the release we most likely want to run as many tests as possible. This environment is not suited for performance testing though so we leave these ones out.

 

Tip
Tip

 

8)     Conclusion

 

Summary

 

 

Everyone talks about segregation now. There is something to it as even in the automated testing world it becomes invaluable at certain stages.

 

Need more insight?
Repository
Repository
Glossary
Glossary
Tags:
Reference
You may also like:
junit5-ordered-tests
JUnit 5 ordered tests
junit5-conditional-execution
JUnit 5 conditional execution
Comments
Be the first to comment.
Leave a comment