Reorganizing Long Test Classes

For the next release of FEST-Swing, the number of automated tests has increased considerably: from 2,677 to 3,000+ (and counting!) Special attention has been paid to the quality of the tests: DRY and readable code, meaningful assertions, and proper naming. Tests are treated with the same TLC (Tender Loving Care) as the rest of the code base.

One important thing was still missing: organization. We have a one-to-one mapping between class and test, resulting in awfully long test classes. For example, all the tests for the class JTreeDriver are contained in JTreeDriverTest, which has 754 lines! 754 lines! When looking at it, I noticed how unmanageable this test class was:

• I lose focus by scrolling up and down, even with a 30-inch monitor
• it’s hard to find tests for a specific method: I have to look for references using my IDE, which can be time-consuming
• it’s hard to figure out if existing tests were covering all possible use cases, especially if they are not grouped by the method they are testing
• waste of resources: not all methods need the same test fixture
• when testing overloaded methods, the names for test methods tend to be too long

I thought that it would be better to create one test-class per method-to-test. To be honest, I wasn’t completely sure it was a good idea, until I read the excellent article "Big Flat Test Cases Suck (and what to do about it)" by Hamlet D’Arcy. Unfortunately, I cannot follow all of Hamlet’s advices, because I use Java and not the Groovy + easyb combo.

Hamlet’s article was a great help! I ended up creating one test-class per method-to-test, as I initially considered. I also reworked the names for the test methods, by adding underscores to improve readability.

As an example, going back to JTreeDriverTest, it originally contained the following methods to test JTreeDriver#expandRow(int):

• shouldExpandRow
• shouldNotDoAnythingIfRowAlreadyExpanded
• shouldThrowErrorWhenExpandingRowInDisabledTree
• shouldThrowErrorWhenExpandingRowInNotShowingTree

and the following methods to test JTreeDriver#expandPath(String):

• shouldExpandPath
• shouldNotDoAnythingIfPathAlreadyExpanded
• shouldThrowErrorWhenExpandingPathInDisabledTree
• shouldThrowErrorWhenExpandingPathInNotShowingTree

I moved those methods to their own test classes, JTreeDriver_expandNodeByRowIndex_Test and JTreeDriver_expandNodeByPath_Test, respectively. I particularly like this naming convention for test classes, because I can find all the tests for a particular method pretty quickly. IMHO, another well-received result is DRY-er method names, since, in this case, the distinction between row index and path is specified in the class name, once, instead of in each test method.

The new test classes have only the following methods:

• should_expand_node
• should_not_do_anything_if_node_already_expanded
• should_throw_error_when_JTree_is_disabled
• should_throw_error_when_JTree_is_not_showing

As far as I can tell, this new organization of test code has effectively solved the problems I mentioned earlier. It also resulted in better method naming. While performing these changes, I found that some use cases were not being tested, and I found a couple of bugs too!

I’m still in the middle of this reorganization. It is a lot of work. It may even delay the release date (originally scheduled for August 3rd.) But I think it’s worth it. I don’t dare to have a formal release if I cannot run the whole test suite successfully. On the meantime, I plan to release a "snapshot" for the users that are waiting for new features or bug fixes.

Oh BTW, I’m not claiming that I invented all the practices presented in this post. I’m only sharing something I learned, hoping it can be useful to others :)

Comments/suggestions are always welcome!

From http://alexruiz.developerblogs.com