Stubs vs Mocks

By Vladimir Khorikov

In this article, I’d like to discuss the differences in using stubs and mocks and show how you can abandon using mocks even in the cases where you need to verify that objects interact with each other correctly.

Stubs vs Mocks: definitions

Before we jump to the actual topic, we need to set some basis and discuss what the definitions of these terms are. The classification of mocks and stubs (as well as fakes and spies) is a bit inconsistent across different authors, but I think the most present, albeit simplified, description would be the following.

  • Mocks are dynamic wrappers for dependencies used in tests. They are programmed with expectations of what exact method calls in what sequence the dependency should receive from a system under test (SUT) in order for the SUT to be deemed working correctly. In .NET, Moq and RhinoMocks are libraries that help create such dynamic wrappers.
  • Stubs are hand-written classes that mimic the dependency’s behavior, but do that with significant shortcuts. For example, a stub for an order repository can store orders in memory and return those orders as a result of search operations instead of querying a real database. Another example would be an email gateway stub that records all emails sent through it. A stub can also be dumb and have only minimal implementation required to satisfy the interface it wraps.

Both stub and mock belong to the notion of test doubles. Test double is a generic term used for indicating some kind of dependency substitution for the test purposes:

Stubs vs Mocks: types of test doubles

Test doubles: mocks and stubs

One could categorize stubs further to fakes and spies, but I personally prefer not to do that as the distinctions between fakes and spies are not as important as the differences between mocks and stubs.

Do we need mocks for testing interactions between objects?

In a typical enterprise software, it is possible to use test doubles only with integration tests and use real classes for unit testing. It is feasible because there is a domain model in such applications, which can be extracted from other parts of your code base and used for unit testing. The domain model usually don’t contain any external dependencies, hence no need for test doubles.

In other types of software, such as 3rd party libraries, you rarely can avoid using some kind of a test double in unit tests.

An example here could be a callback object, methods of which the SUT has to invoke in an exact sequence. A common way to verify the SUT’s behavior in such situations is to use mocks. Programming a mock for a particular order of operations and verifying that order afterward helps us to check that the SUT interacts with the callback correctly.

But do we really need mocks to do that? Can stubs do a better job? Let’s take a code example that relies on mocks and see if there are any benefits in transforming it into a version that uses stubs.

[Fact]

public void Parser_parses_xml_in_correct_order()

{

    // Arrange : input values

    string xml = “<outer><inner /></outer>”;

    var parser = new Parser();

 

    // Arrange : record expectations

    var mocks = new MockRepository();

    IHandler handler = mocks.CreateMock<IHandler>();

    handler.StartDocument();

    handler.StartElement(“outer”);

    handler.StartElement(“inner”);

    handler.EndElement(“inner”);

    handler.EndElement(“outer”);

    handler.EndDocument();

    mocks.ReplayAll();

 

    // Act

    parser.ParseXml(xml, handler);

 

    // Assert

    mocks.VerifyAll();

}

In the sample above, we are testing how the Parser class interacts with IHandler. In order for the Parser to be correct, it should call specific methods of the handler in a particular sequence.

The mock does a pretty good job here, but the code sample has the flaw we discussed in the previous article: the test relies on the implementation details of the Handler class and thus is prone to refactoring.

Let’s see how we can write the same test using a hand-written stub. First, we need the stub itself:

public class HandlerStub : IHandler

{

    private List<Tuple<Action, string>> actionsCalled = new List<Tuple<Action, string>>();

 

    public void StartDocument()

    {

        actionsCalled.Add(new Tuple<Action, string>(Action.DocumentStarted, null));

    }

 

    public void StartElement(string elementName)

    {

        actionsCalled.Add(new Tuple<Action, string>(Action.ElementStarted, elementName));

    }

 

    /* Other methods */

 

    internal enum Action

    {

        DocumentStarted,

        ElementStarted,

        ElementEnded,

        DocumentEnded

    }

}

The stub basically records all the interactions that come from the outside world and provides methods to validate those interactions.

Now, we can use this stub in our test:

[Fact]

public void Parser_parses_xml_in_correct_order()

{

    // Arrange

    string xml = “<outer><inner /></outer>”;

    var parser = new Parser();

    var handlerStub = new HandlerStub();

 

    // Act

    parser.ParseXml(xml, handlerStub);

 

    // Assert

    handlerStub.WasCalled()

        .WithStartDocument()

        .WithStartElement(“outer”)

        .WithStartElement(“inner”)

        .WithEndElement(“inner”)

        .WithEndElement(“outer”)

        .WithEndDocument();

}

As you can see, we achieved the same result with no mocks whatsoever.

The distinction between the former and the latter versions of the test might seem subtle, but it is rather significant. With mocks, you have to mimic the IHandler interface with every test that uses it, which leads to code duplication and thus to brittle design. Every change in IHandler would cause cascade failures throughout all tests that use it.

With stubs, you pack the fragility into a single stub implementation. You abstract your tests from the internal details of the dependency. If you change the Handler interface, you need to adjust the stub only once; you don’t have to touch any tests that use it.

That’s the power of a reliable test suite. Hand-written stubs help create unit tests that verify the end result of an interaction without knowing the internal implementation details of that interaction. They help us adhere to the most important TDD rule, which is keeping the tests one level of abstraction above the code they check.

I have to make a note here. Although I strongly advocate you prefer stubs over mocks, there are situations where you are better off choosing mocks. In the cases where you need to create only a single unit test that uses a dependency, there are no effective differences between a test that relies on mocks and the one that employs stubs.

In both cases, you would need to change the code only once should a refactoring occur. Because of that, mocks would be a preferable choice as they require less up-front effort than stubs. But whenever you see you start having more than one test that substitute the same dependency using mocks, you should switch to a hand-written stub instead.

If you enjoyed this article, be sure to check out my Pragmatic Unit Testing Pluralsight course too.

Summary

Even in situations where you need to test the correctness of the interactions between classes, you can check the end result of those interactions, not the behavior that led to that result.

Stubs help us do that. Although using them require more up-front effort, it pays off greatly. Tests that rely on stubs are less prone to refactoring and thus lay the ground for building a solid unit test suite.

The source code for the code sample can be found here.

Other articles in the series





  • http://rpajak.com/ Pellared
    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      The classification of spies and stubs is a bit inconsistent. Indeed, Gerard Meszaros in his book defines the classes like the one I’m using in my code sample as spies. At the same time, Fowler gives a broader definition:

      “Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it ‘sent’, or maybe only how many messages it ‘sent’.”
      Source: http://martinfowler.com/articles/mocksArentStubs.html

      I agree that Spy would be more precise definition for what I’m doing in that sample, it’s just that I think the differences between stubs and spies are not as important as the differences between stubs/spies and mocks.

      Regarding your code sample – that’s an interesting example of using mocks! Is does solve the problem of design brittleness, but I think this approach is more akin to creating stubs/spies than to using mocks in a classic sense. Using mocks usually implies you mock a dependency in every unit test over and over again while the HandlerSpy you introduced removes that repetition just like a classic spy would.

      • DAXx85

        Good reply, pal, as well as this article is!
        I appreciate this meaningful distiction between mocks and stubs. Although I do not have a huge experience in writing tedious Unit Tests with all the heavy stuff mocked, it always seemed questionable to me to see guys using some black box mocking frameworks instead of writing lightweight and smooth stubs for dependencies they wanted to abstract from the test case.
        Keep on writing those articles in a simple, approachable manner with a clear message inside.

        • http://enterprisecraftsmanship.com/ Vladimir Khorikov

          Indeed, the right tool for the right job. Thank you for your kind words! I do try to keep simplicity as the main attribute in my articles, I’m glad you noticed that!

  • Jes Slow

    Hi, where does your HandlerStub get methods such as WasCalled() and and WithStartDocument()? I’m not a Java programmer. Do you havefull code on github or something?

  • Linar Sungatullin

    Hi. I read in one book about unit tesing, that stubs are fake objects, that return some state(e.g. user name) and do nothing more. And mocks are fake objects too, but they become mocks when we verifying their behaviour in assert section.

    For example in Moq: when we create some var mock = Mock(); – it is a stub and when we call mock.Verify(…) it becomes a mock. And there is advice not to use more than one Verify(it means mock) in one test.

    What do you think about it? Anyway you articles about unit testing are awesome! 🙂

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Thanks Linar!

      Yes, I heard about such definition too. Basically, if you only fake side-effect free methods (queries) on an object, it’s a stub. Faking methods with side-effects (commands) makes a test double a mock.

      This definition makes sense as well. And it’s more abstract as it doesn’t depend on the actual implementation you choose: you can either use Moq library to implement a mock or you can create a hand-written class. As I mentioned, the classification here is a bit inconsistent, each author brings their own view on the topic.

      Regarding a single Verify() call per unit test guideline, I think it’s more related to the granularity of unit tests than to the subject of mocks versus stubs. It’s a good practice to test only one thing at a time, and multiple Verify calls is a strong sign a test doesn’t adhere to this practice.