Integration Tests: Avoiding False Dichotomies
Integration testing is getting a lot of attention right now as Arquillian starts to emerge from the cocoon and the Forge videos get tweeted up. The last guys to attempt to tell us what an integration test was (as they were claiming to be the Prometheans who were unlocking them) were the Spring camp. While Spring tests were useful, they laid up pretty far from the hole. Likewise, the book Growing Object Oriented Software Guided by Tests end their book with one of the most delusional self-congratulatory encomia in history: according to them, JMock saved development. Reality: in 2011 we still cannot really do integration tests. Only question left is will we ever be able to.
So there are a lot of ways to look at integration tests. One of the most common is ‘everything that can‘t be unit tested.‘ The problem with this definition is that it doesn‘t contain the outer realm: if you did any tests that spanned a unit, you get to check the box if this is your navigational impetus. Kind of like describing space as ‘the part of the universe where there‘s no atmosphere.‘ That said, starting with TestNG‘s Cedric, I do think that people who burst in on the silly ‘just isolate everything into units and leave me alone‘ cadre did us all a good turn. Remember what it was like to even do a test in JUnit 3.8 that had any setup that had to be done once for all the tests?? Ugh.
But I want to back up from the tactical level and stay in the more cosmological realm.
Another answer to what an integration test is is a test that exercises collaborations. This is where the Spring testing guys came down: we are providing a way of wiring collaborators and giving you a way to then test that they are doing what they ought when working together This is a decent goal, but there are some downsides to this approach too. One is that it wakes up one day so impressed with itself, it imagines that before them, there was no way tell if anything worked at all. In fact, I think that Spring testing promoted a specious view of when it was time to ‘integrate‘ at the peril of the underlying models: there are quite often classes that are collaborators, and though wiring collaborators together through a bean container has some advantages, doing so without is not impossible, or necessarily bad. Spring was definitely a realm with few notions of gradations, but I could make an argument that a Money class needs to collaborate with a Currency class, but wiring them together through a container would be madness.
My dream for integration testing is tied up with my dream for component-based development: that it will not have an outside, and that the speed of testing to any depth or layer will be uniformly rapid. What does that mean? I want everything to be tested through one mechanism, and that includes bindings in a facelet, conversions, validations, outbound messages, etc. The dumbest Magoo moment in the history of testing was the dopes who were running around bragging about getting to 100 on their coverage reports. If we had a corresponding score that showed that was 100% of 100%, it might be impressive, but of course we don‘t. (Then again, when you consider that amidst 12% unemployment the California Prison Guards Union just won a concession to be paid from the second they get out of their car, the most witless in our profession look like Hercules.)
Testing tools and frameworks are great, but at some point, the language and the tools we are using need to become test collaborators, and this is rarely the case. I like JSF a lot, but it's not particularly testable, and it could have done a lot more to make itself more testable. I blogged before about the fact that Objective-C did the so-called Hollywood Principle years before Spring did (and probably better): things are super simple to test when anything can simply declare itself a delegate and implement an interface.
Differential definition is the norm in the Java world (generals are always fighting the last war). Nothing wrong with it as part of a broader view.
Will we get there with Arquillian and Forge and the like? I hope so.