Mock Frameworks vs. Microsoft Fakes

Should you mock it or fake it? That’s a question you probably find yourself asking when designing a software testing strategy.

It isn't always easy to decide which mock framework is best for testing your code. For developers using Microsoft Visual Studio, the most basic choice is whether to use VS's built-in Microsoft Fakes framework or a third-party open-source framework such as Moq or NSubstitute. In this post, we'll take a look at what makes MS Fakes stand out from other mock frameworks and when Fakes may or may not be your best choice for testing.

The Real Fake Story

First, though, some background. What are mock frameworks and why would you use them? In many ways, mock frameworks are simply extensions of traditional testing techniques, updated for object-oriented programming.

The basic problem that mock frameworks are designed to solve is this: When you're testing code that is still in development, you need to be able to test its interaction with dependencies, with outside applications, and with system resources — but it is not always possible to do so. Very often, the dependencies are still under development or have not been created, and even when your code is interacting with external programs or system components, the tests may require a set response (such as a specific time of day or error code), which the application or resource cannot be counted on to supply.

A mock framework allows you to supply realistic emulations of the required interactions by means of mocks, stubs, and shims. (Note that these terms do not always have clear, agreed-upon definitions, and are sometimes used interchangeably. Here, we will try to remain consistent with reasonably widespread definitions and use.)

Making the Right Kind of Mockery

Mocks interact with the code being tested by means of interfaces. This means that in order to use mocks for testing, you must write code that is fully interface-compliant. A mock stands in for the object which it represents. From the point of view of the unit being tested, there should be no difference between a mock and the actual object.

A mock, however, does not need to duplicate the internal operations of the object that it represents. Instead, it can return hard-coded responses (always sending back "True" or "Smith, J.Q.," for example), or it can contain logic designed to test the unit in more sophisticated ways (by using assertions, or by emulating complex behavior on the part of a client, for instance).

Stub It

A stub is also interface-based and is in many other ways similar to a mock. In practice, the distinction between them is not always clear. In general, however, the main function of a stub is to stand in for an object by returning hard-coded values as if it were that object. A stub may include simple assertions, but it typically will not include the kind of complex test logic that is often used by mocks.

Fitting in a Shim

A shim functions much like a stub, returning set values and testing against assertions — but it does so in situations where interaction with the resource or application is not interface-based. In effect, it slides between the code under test and the resource, emulating the resource's response when required for testing. If the target resource is not fully interface-compliant, you must use a shim rather than a mock or stub.

The Microsoft Fakes Approach

Where does the Microsoft Fakes framework fit into this overall system?

Visual Studio Integration

The first and perhaps most obvious feature that makes MS Fakes stand out is that it is fully integrated into Microsoft Visual Studio. This means that you can do such things as automatically generate stub classes for every interface in an assembly simply by selecting "Add Fakes Assembly" from the context menu for a reference to that assembly. Visual Studio recognizes stub classes and treats them as such. In debugging, for example, it ignores the generated code forming the stub structure and looks strictly at the programmer-created code that the stub contains. The process of generating and using shims is integrated into Visual Studio in a similar manner.

Fakes but No Mocks

The second (and somewhat controversial) distinctive feature of Microsoft Fakes is that it generates stubs and shims, but not full mocks. This does not mean that you cannot provide a stub with the kind of features found in mock frameworks such as Moq, but it does mean that you have to specifically add them and take care of any housekeeping by hand. Depending on the kind of tests that you are doing and the level of complexity of the tests, this can result in a significant amount of programming overhead.

Good Shims and Bad

The third (and also controversial) feature of Microsoft Fakes is its use of shims. Shims are necessary if you want to incorporate any non-interface-compliant interactions, of course, but to some critics of their use, that is exactly the problem. Code that bypasses interfaces, they say, is sloppy code, and more to the point, it greatly increases the probability that at some point in the future, it will lead to broken dependencies. Making it easy to generate shims for testing, they fear, will simply encourage bad programming habits. Defenders of shim-based testing, however, point out that non-interface interactions are sometimes unavoidable, particularly when dealing with external applications that do not provide an API.

Microsoft's Recommendations

Microsoft's own recommendations regarding stubs and shims take these considerations into account. Their basic recommendation is that dependencies which you create as part of your codebase should be fully interface-compliant and that you use stubs in place of those dependencies during testing. Shims should be used where stubs cannot be used - with external resources that include static, non-virtual, private or sealed methods, or which otherwise do not include an API that can be used for testing with stubs.

Microsoft's documentation brings up another consideration regarding shims: They work by creating detours (from the real method being called to the shim's method) in your application code. Since both the creation and deletion of detours occurs during run-time, it can slow down test execution. There is no equivalent delay with stubs.

Which Should You Choose?

So, which is best for you — a third-party mock framework or Microsoft Fakes? In many ways, that depends on what kind of testing you intend to do. If you want in-depth testing of complex dependency interactions, a third-party framework with full mocking features might be more suitable. However, if complex testing is less important than being able to quickly set up the framework for testing of basic dependency interactions, or many of your dependencies involve external applications and system resources which require shims rather than stubs or mocks, Microsoft Fakes may be the best choice.