The Occasionally Failing Test

This piece of code is part of a test that runs a scenario and checks that the appropriate errors are logged. Very occasionally, this test would fail, and it would be nearly impossible to figure out why.

I’ve extracted the offending code from the test. ReadFromWebSocket returns a Task<string>, since that isn’t obvious from the code. Do you see the error?

Think about it. What is this doing?

This is reading from a socket, and there is absolutely no guarantee about the amount of data that will go over the wire at any particular point. Because this test assumes that the entire string will be read in a single call from the socket, if the expected value we are looking is actually going to be returned in two calls, we’ll miss it — and this will never return, leading to sadness and worry everywhere — at least everywhere that we care about our tests!

The fix was to remember the previous values and compare to all the data read from the socket, not to just the values that were returned in the last call.