Spocklight: Optimize Run Order Test Methods
Spock is smart, and we can tell Spock to re-run failing methods before successful methods, with the fastest ones first.
Join the DZone community and get the full member experience.
Join For FreeSpock is able to change the execution order of test methods in a specification. We can tell Spock to re-run failing methods before successful methods. And if we have multiple failing or successful tests, then first run the fastest methods, followed by the slower methods. This way when we re-run the specification we immediately see the failing methods and could stop the execution and fix the errors. We must set the property optimizeRunOrder
in the runner
configuration of the Spock configuration file. A Spock configuration file with the name SpockConfig.groovy
can be placed in the classpath of our test execution or in our USER_HOME/.spock
directory. We can also use the Java system property spock.configuration
and assign the filename of our Spock configuration file.
In the following example we have a specification with different methods that can be successful or fail and have different durations when executed:
package com.mrhaki.spock
import spock.lang.Specification
import spock.lang.Subject
class SampleSpec extends Specification {
@Subject
private Sample sample = new Sample()
def "spec1 - slowly should return name property value"() {
given:
sample.name = testValue
expect:
sample.slowly() == testValue
where:
testValue = 'Spock rules'
}
def "spec2 - check name property"() {
given:
sample.name = testValue
expect:
sample.name == testValue
where:
testValue = 'Spock is gr8'
}
def "spec3 - purposely fail test at random"() {
given:
sample.name = testValues[randomIndex]
expect:
sample.name == testValues[0]
where:
testValues = ['Spock rules', 'Spock is gr8']
randomIndex = new Random().nextInt(testValues.size())
}
def "spec4 - purposely fail test slowly"() {
given:
sample.name = 'Spock is gr8'
expect:
sample.slowly() == 'Spock rules'
}
def "spec5 - purposely fail test"() {
given:
sample.name = 'Spock rules'
expect:
sample.name == 'Spock is gr8'
}
}
class Sample {
String name
String slowly() {
Thread.sleep(2000)
name
}
}
Let's run our test where there is no optimized run order. We see the methods are executed as defined in the specification:
Next we create a Spock configuration file with the following contents:
runner {
println "Optimize run order"
optimizeRunOrder true
}
If we re-run our specification and have this file in the classpath we already see the order of the methods has changed. The failing tests are at the top and the successful tests are at the bottom. The slowest test method is last:
Another re-run has optimized the order by running the slowest failing test after the other failing tests.
Spock keeps track of the failing and successful methods and their execution time in a file with the specification name in theUSER_HOME/.spock/RunHistory
directory. To reset the information we must delete the file from this directory.
Written with Spock 1.0-groovy-2.4.
Published at DZone with permission of Hubert Klein Ikkink, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments