I took advantage of the holiday break to completely revamp TestNG’s HTML reports, something I’ve been meaning to do for a very long time but never found the time to put at the top of my TODO list. I wrote the original reports pretty much with the very first version of TestNG, around 2004, and I hardly touched them since then. My intention with this rewrite was not just to revamp them visually but also technically, so that I could give myself as much freedom as possible to improve them from this point on.
Here are some notes I took during this process.
Content and appearance
First of all, here are the new reports. They show all the suites on the same page with a banner at the top, a navigator pane on the left hand side (which always stays visible) and a main panel on the right hand side, which shows all the information you requested depending on which item you clicked in the navigator. Pretty straightforward.
If you take a look at the source, you will see that it hardly contains any HTML: it’s mostly made of <div> and <span>. I didn’t really set out to do this initially, it’s just that whenever I used HTML elements, I inherited undesirable CSS attributes (margins, paddings, etc…) which I ended up resetting manually, so after a while, it just seemed much easier to use divs and spans knowing that they start with a clean CSS slate.
Once I had the templating code working, I started converting the Java code over to it but… something felt wrong. I quickly gained the impression that this was actually a step backward compared to doing the generation 100% from Java, and the reason quickly became apparent to me: the Java compiler.
The problem with using templates is that you get very little help from the tools. Some template libraries allow you to include other templates, which can reduce the repetition, but it’s still very easy to make simple typos or being forced to copy/paste a lot. In contrast, my Java hierarchies makes it quite trivial to, say, add a new panel on the right hand side with a link to the left. Implement the right class, declare it and it will automatically work, with the right defaults (CSS selectors) and a lot of the logic validated by the compiler.
This comfort was too good to pass up so I scrapped my template idea (my mustache.js implementation is still around, though, and I might use it some time in the future) and I stuck to 100% Java generation.
Chrome’s development environment is also quite superb (I’m sure other browsers’ is just as good). In a nutshell, you have access to pretty much the same range of support that Eclipse or IDEA provides in Java: breakpoints, inspection and modification of variables, and even CSS debugging, which I’ve found invaluable to track down unexpected CSS behaviors (I came across quite a few).
- Open http://testng.org/new in Chrome.
- Right click anywhere on the page and select "Inspect element". This will open the document explorer at the bottom.
- With the focus in the document explorer, type ESC, which will open the REPL.
- Type $('.navigator-suite-content').hide().
- This will collapse the content under both suites.
You can see how easy it is to debug this kind of code.
If you select the "Times" link of the first test suite in the Navigator pane, you will see the following table:
Here are my main take away points:
- CSS still feels like black magic to me. The theory is trivial on paper but in reality, I often come across results that just don’t make sense. With the help of modern CSS debuggers, it’s a little bit easier to find out what is happening and why (I especially like Chrome’s “Computed style” which gives you a list of all the attributes that were derived from .css files), but there are still times where I feel completely helpless trying to find out why something is not how it should be.
- I’m a bit concerned with the size of the reports: I was quite surprised to find out that the reports you’re looking at is one meg in size. Admittedly, it shows over five hundred test methods, but the fact that it contains all the panels, even those the user might not be interested in, is a potential place for improvement. I might decide to put each individual pane in its own file.
Finally, a call for help: I welcome any feedback on how to improve the CSS of these new reports, so if you are so inclined, feel free to improve the look of these new reports and share your improvements with me. I will be very grateful.
Happy new year!