Execute tests in random order

Filed under: Java, Maven, TDD, Test automation, — Tags: JUnit, Random, Surefire, runOrder — Thomas Sundberg — 2012-05-03

It works on my machine!

Ever heard that from a developer? I have, and it happens that the reason is that it actually works on their machine. It may also be the case that the order in which tests are executed matters. Test classes depends on each other and the order they are executed in is important. The solution is to execute the tests in random order so that any dependencies between tests are found and can be removed.

You can specify the order test classes are fed to Surefire in Maven. The order in which the tests are executed in a test class is not possible to influence as far as I know. You have to set a configuration runOrder in the Surefire plugin. You may specify different run orders and the one I would use to find dependent test is random

<?xml version="1.0" encoding="UTF-8"?>
<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>se.sigma.educational</groupId>
    <artifactId>randon-run-order</artifactId>
    <version>1.0</version>
    <packaging>jar</packaging>
    <build>
        <finalName>executable-example</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.12</version>
                <configuration>
                    <runOrder>random</runOrder>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

This will ensure that the test classes are executed in different order and any dependencies between them will eventually pop up.

Resources



(less...)

Pages

About
Events
Why

Categories

Agile
Automation
BDD
Clean code
Continuous delivery
Continuous deployment
Continuous integration
Cucumber
Culture
Design
Executable specification
Gradle
Guice
J2EE
JUnit
Java
Linux
Load testing
Maven
Mockito
Pair programming
PicoContainer
Programming
Public speaking
Quality
Recruiting
Requirements
Scala
Selenium
Software craftsmanship
Software development
Spring
TDD
Teaching
Technical debt
Test automation
Tools
Web
Windows
eXtreme Programming

Authors

Thomas Sundberg
Adrian Bolboaca

Archives

Meta

rss RSS