Over a million developers have joined DZone.

Why You Need to Make Your Tests Fail

· Agile Zone

Learn more about how DevOps teams must adopt a more agile development process, working in parallel instead of waiting on other teams to finish their components or for resources to become available, brought to you in partnership with CA Technologies.

Test Driven Development (TDD) has many benefits. For start it’s a design methodology that help avoiding “Analysis paralysis” and make sure that you only have the needed code to solve a problem.
Yesterday I found another benefit of writing the tests before the code – you get to see them fail!
A while back I wrote about another shortcoming of MSTest – how it does not have any built in facility to test against expected exception message.

Unknown to me at the time there was a bug hidden in the code I’ve published – see if you can spot it:

public static class MyAssert
{
    public static void BadThrows;<T>(Action action) where T : Exception
    {
        try
        {
            action();

            Assert.Fail("Exception of type {0} should be thrown.", typeof(T));
        }
        catch (T exception)
        {
     // All is well!
        }
    }
}

BTW this example was simplified for your viewing pleasure - I want to make a point here, not stick to the facts. have you found the bug yet?

Consider the following test:

[TestMethod]
public void ThisTestWouldNeverFail()
{
    MyAssert.BadThrows<exception>(() => Dummy.SomeMethod());
}
Have you guessed it? (hint: its in the test name)

That's right! this test would never have failed. If we take a closer look at BadThrows we’ll see the problem.

MSTest just like any other .NET unit testing framework throws an exception when assertions fails. In this case because we’re expecting an exception of type Exception any other exception thrown including the AssertFailedException would be caught – causing the test to pass.

That’s right – because the user expects an exception of type”Exception” to be thrown plus a minor bug the test is completely useless – if you want to see the working code just head to this post.

I did not find this bug mainly because I don’t like to throw and assert for Exception seems like a bad practice and in retrospective I should have avoided my co-workers from doing so by throwing exception of my own in case they try to do it.

The guy that did find this bug found it because he wrote a test and wanted to see it fail. That’s right it wasn’t enough for him to know that his code made the test pass – he had to make sure that it failed first – some people…
A quick search in our code found multiple tests that did exactly that which means two things:
  1. These test were not written using TDD
  2. The authors did not even bother to see the tests fail
After fixing the bug we found one test that started failing – which means that not only was the test wrong – so was the code it was testing.

I’ve talked to the team, explained why throwing “Exception” is not a best practice and why tests need to be “tested” by seeing them fail and we’ve fixed the tests and the code.

I think I learnt quite a lot from this experience, I know now who does TDD and who doesn’t and I found a new problem that occurs when developers write their tests after the code.

Happy coding…

Discover the warning signs of DevOps Dysfunction and learn how to get back on the right track, brought to you in partnership with CA Technologies.

Topics:

Published at DZone with permission of Dror Helper, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}