Over a million developers have joined DZone.

Spocklight: Optimize Run Order Test Methods

Spock is smart, and we can tell Spock to re-run failing methods before successful methods, with the fastest ones first.

· DevOps Zone

Discover how to optimize your DevOps workflows with our cloud-based automated testing infrastructure, brought to you in partnership with Sauce Labs

Spock is able to change the execution order of test methods in a specification. We can tell Spock to re-run failing methods before successful methods. And if we have multiple failing or successful tests, then first run the fastest methods, followed by the slower methods. This way when we re-run the specification we immediately see the failing methods and could stop the execution and fix the errors. We must set the property optimizeRunOrder in the runner configuration of the Spock configuration file. A Spock configuration file with the name SpockConfig.groovy can be placed in the classpath of our test execution or in our USER_HOME/.spock directory. We can also use the Java system property spock.configuration and assign the filename of our Spock configuration file.

In the following example we have a specification with different methods that can be successful or fail and have different durations when executed:

package com.mrhaki.spock

import spock.lang.Specification
import spock.lang.Subject

class SampleSpec extends Specification {

    @Subject
    private Sample sample = new Sample()

    def "spec1 - slowly should return name property value"() {
        given:
        sample.name = testValue

        expect:
        sample.slowly() == testValue

        where:
        testValue = 'Spock rules'
    }

    def "spec2 - check name property"() {
        given:
        sample.name = testValue

        expect:
        sample.name == testValue

        where:
        testValue = 'Spock is gr8'
    }

    def "spec3 - purposely fail test at random"() {
        given:
        sample.name = testValues[randomIndex]

        expect:
        sample.name == testValues[0]

        where:
        testValues = ['Spock rules', 'Spock is gr8']
        randomIndex = new Random().nextInt(testValues.size())
    }

    def "spec4 - purposely fail test slowly"() {
        given:
        sample.name = 'Spock is gr8'

        expect:
        sample.slowly() == 'Spock rules'
    }

    def "spec5 - purposely fail test"() {
        given:
        sample.name = 'Spock rules'

        expect:
        sample.name == 'Spock is gr8'
    }
}

class Sample {
    String name

    String slowly() {
        Thread.sleep(2000)
        name
    }
}

Let's run our test where there is no optimized run order. We see the methods are executed as defined in the specification:

Image title

Next we create a Spock configuration file with the following contents:

runner {
    println "Optimize run order"
    optimizeRunOrder true
}

If we re-run our specification and have this file in the classpath we already see the order of the methods has changed. The failing tests are at the top and the successful tests are at the bottom. The slowest test method is last:

Image title

Another re-run has optimized the order by running the slowest failing test after the other failing tests.

Image title

Spock keeps track of the failing and successful methods and their execution time in a file with the specification name in theUSER_HOME/.spock/RunHistory directory. To reset the information we must delete the file from this directory. 

Written with Spock 1.0-groovy-2.4.

Download “The DevOps Journey - From Waterfall to Continuous Delivery” to learn learn about the importance of integrating automated testing into the DevOps workflow, brought to you in partnership with Sauce Labs.

Topics:
spock ,java ,groovy ,testing ,optimize run order test ,test optimization ,spocklight

Published at DZone with permission of Hubert Klein Ikkink, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}