Asynchronous Functional Testing

I previously wrote about writing SQL tests. Another difficult-to-test scenario involves asynchronous processes. If a method-under-test has threading, getting the timing right for the TestCase asserts is difficult. Typically, the method will return control to the test class prior to completing the logic that is being tested. The test class then immediately starts processing the asserts. These tests are flaky at best or completely broken at worst.

One way to avoid testing asynchronous scenarios is to simply test the units individually. I am going to take a moment to discuss my view on unit tests.

So with that in mind, let’s discuss a technique for testing asynchronous processes. Please review the following code.

def methodUnderTest(def lowerString, Closure callback) {
  Thread.start {
    Thread.sleep(500)
      callback(lowerString.toUpperCase())
  }
}
  
def actual
methodUnderTest("async", {x -> actual = x})
 
assert actual == "ASYNC"
println 'Success' //does not print

This is a simple use case, but is easy to understand. The method-under-test accepts a string and a callback function. The threaded logic simply uppers the string and calls the callback with the new value. Thread.sleep(500) simulates a process taking a little time to complete.

The result of running the above is as follows:

Assertion failed: 
 
assert actual == "ASYNC"
       |      |
       null   false

Try running that in GroovyConsole. If you were to comment on the Thread.start{}, the assert would pass and “Success” would print. That’s great, but we need it to pass in a multi-threaded implementation. We can take care of that with a little test helper. Review the AsyncAssert class in the following example.

class AsyncAssert {
  public static void run(Closure closure) {
    run(5, closure)
  }
 
  public static void run(int timeout, Closure closure) {
    def timestamp = System.currentTimeMillis()
 
    while(true){
      try {
        closure()
        break
      }
      catch(Throwable all) {
        Thread.sleep(250)
        def processTime = (System.currentTimeMillis() - timestamp) / 1000
        if (processTime > timeout ) {
          throw all
        }
      }
    }
  }
}
 
def methodUnderTest(def lowerString, Closure callback) {
  Thread.start {
    callback(lowerString.toUpperCase())
  }
}
 
def actual
methodUnderTest("async", {x -> actual = x})
 
AsyncAssert.run {
  assert actual == "ASYNC"
  println 'Success'
}

The run method in AsyncAssert executes the asserts and catches all exceptions. It checks if the timeout has lapsed in the exception handler. If not, then the asserts are run again. If the timeout has passed, it will throw the exception (typically an AssertionException of some sort). If the asserts pass without an exception, the loop breaks out and the test passes.

So yes, asynchronous tests will run a little slower than your typical unit tests. For me, the piece of mind that the process works is worth a couple extra seconds. How do some of you test your asynchronous processes?