Stubs vs Mocks



In this article, I’d like to discuss the differences in using stubs and mocks and show how you can abandon using mocks even in the cases where you need to verify that objects interact with each other correctly.

Stubs vs Mocks: definitions

Before we jump to the actual topic, we need to set some basis and discuss what the definitions of these terms are. The classification of mocks and stubs (as well as fakes and spies) is a bit inconsistent across different authors, but I think the most present, albeit simplified, description would be the following.

  • Mocks are dynamic wrappers for dependencies used in tests. They are programmed with expectations of what exact method calls in what sequence the dependency should receive from a system under test (SUT) in order for the SUT to be deemed working correctly. In .NET, Moq and RhinoMocks are libraries that help create such dynamic wrappers.
  • Stubs are hand-written classes that mimic the dependency’s behavior, but do that with significant shortcuts. For example, a stub for an order repository can store orders in memory and return those orders as a result of search operations instead of querying a real database. Another example would be an email gateway stub that records all emails sent through it. A stub can also be dumb and have only minimal implementation required to satisfy the interface it wraps.

Both stub and mock belong to the notion of test doubles. Test double is a generic term used for indicating some kind of dependency substitution for the test purposes:

Stubs vs Mocks: types of test doubles

Test doubles: mocks and stubs

One could categorize stubs further to fakes and spies, but I personally prefer not to do that as the distinctions between fakes and spies are not as important as the differences between mocks and stubs.

Do we need mocks for testing interactions between objects?

In a typical enterprise software, it is possible to use test doubles only with integration tests and use real classes for unit testing. It is feasible because there is a domain model in such applications, which can be extracted from other parts of your code base and used for unit testing. The domain model usually don’t contain any external dependencies, hence no need for test doubles.

In other types of software, such as 3rd party libraries, you rarely can avoid using some kind of a test double in unit tests.

An example here could be a callback object, methods of which the SUT has to invoke in an exact sequence. A common way to verify the SUT’s behavior in such situations is to use mocks. Programming a mock for a particular order of operations and verifying that order afterward helps us to check that the SUT interacts with the callback correctly.

But do we really need mocks to do that? Can stubs do a better job? Let’s take a code example that relies on mocks and see if there are any benefits in transforming it into a version that uses stubs.

[Fact]

public void Parser_parses_xml_in_correct_order()

{

    // Arrange : input values

    string xml = “<outer><inner /></outer>”;

    var parser = new Parser();

 

    // Arrange : record expectations

    var mocks = new MockRepository();

    IHandler handler = mocks.CreateMock<IHandler>();

    handler.StartDocument();

    handler.StartElement(“outer”);

    handler.StartElement(“inner”);

    handler.EndElement(“inner”);

    handler.EndElement(“outer”);

    handler.EndDocument();

    mocks.ReplayAll();

 

    // Act

    parser.ParseXml(xml, handler);

 

    // Assert

    mocks.VerifyAll();

}

In the sample above, we are testing how the Parser class interacts with IHandler. In order for the Parser to be correct, it should call specific methods of the handler in a particular sequence.

The mock does a pretty good job here, but the code sample has the flaw we discussed in the previous article: the test relies on the implementation details of the Handler class and thus is prone to refactoring.

Let’s see how we can write the same test using a hand-written stub. First, we need the stub itself:

public class HandlerStub : IHandler

{

    private List<Tuple<Action, string>> actionsCalled = new List<Tuple<Action, string>>();

 

    public void StartDocument()

    {

        actionsCalled.Add(new Tuple<Action, string>(Action.DocumentStarted, null));

    }

 

    public void StartElement(string elementName)

    {

        actionsCalled.Add(new Tuple<Action, string>(Action.ElementStarted, elementName));

    }

 

    /* Other methods */

 

    internal enum Action

    {

        DocumentStarted,

        ElementStarted,

        ElementEnded,

        DocumentEnded

    }

}

The stub basically records all the interactions that come from the outside world and provides methods to validate those interactions.

Now, we can use this stub in our test:

[Fact]

public void Parser_parses_xml_in_correct_order()

{

    // Arrange

    string xml = “<outer><inner /></outer>”;

    var parser = new Parser();

    var handlerStub = new HandlerStub();

 

    // Act

    parser.ParseXml(xml, handlerStub);

 

    // Assert

    handlerStub.WasCalled()

        .WithStartDocument()

        .WithStartElement(“outer”)

        .WithStartElement(“inner”)

        .WithEndElement(“inner”)

        .WithEndElement(“outer”)

        .WithEndDocument();

}

As you can see, we achieved the same result with no mocks whatsoever.

The distinction between the former and the latter versions of the test might seem subtle, but it is rather significant. With mocks, you have to mimic the IHandler interface with every test that uses it, which leads to code duplication and thus to brittle design. Every change in IHandler would cause cascade failures throughout all tests that use it.

With stubs, you pack the fragility into a single stub implementation. You abstract your tests from the internal details of the dependency. If you change the Handler interface, you need to adjust the stub only once; you don’t have to touch any tests that use it.

That’s the power of a reliable test suite. Hand-written stubs help create unit tests that verify the end result of an interaction without knowing the internal implementation details of that interaction. They help us adhere to the most important TDD rule, which is keeping the tests one level of abstraction above the code they check.

I have to make a note here. Although I strongly advocate you prefer stubs over mocks, there are situations where you are better off choosing mocks. In the cases where you need to create only a single unit test that uses a dependency, there are no effective differences between a test that relies on mocks and the one that employs stubs.

In both cases, you would need to change the code only once should a refactoring occur. Because of that, mocks would be a preferable choice as they require less up-front effort than stubs. But whenever you see you start having more than one test that substitute the same dependency using mocks, you should switch to a hand-written stub instead.

If you enjoyed this article, be sure to check out my Pragmatic Unit Testing Pluralsight course too.

Summary

Even in situations where you need to test the correctness of the interactions between classes, you can check the end result of those interactions, not the behavior that led to that result.

Stubs help us do that. Although using them require more up-front effort, it pays off greatly. Tests that rely on stubs are less prone to refactoring and thus lay the ground for building a solid unit test suite.

The source code for the code sample can be found here.

Other articles in the series

Share




  • http://rpajak.com/ Pellared
    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      The classification of spies and stubs is a bit inconsistent. Indeed, Gerard Meszaros in his book defines the classes like the one I’m using in my code sample as spies. At the same time, Fowler gives a broader definition:

      “Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it ‘sent’, or maybe only how many messages it ‘sent’.”
      Source: http://martinfowler.com/articles/mocksArentStubs.html

      I agree that Spy would be more precise definition for what I’m doing in that sample, it’s just that I think the differences between stubs and spies are not as important as the differences between stubs/spies and mocks.

      Regarding your code sample – that’s an interesting example of using mocks! Is does solve the problem of design brittleness, but I think this approach is more akin to creating stubs/spies than to using mocks in a classic sense. Using mocks usually implies you mock a dependency in every unit test over and over again while the HandlerSpy you introduced removes that repetition just like a classic spy would.

      • DAXx85

        Good reply, pal, as well as this article is!
        I appreciate this meaningful distiction between mocks and stubs. Although I do not have a huge experience in writing tedious Unit Tests with all the heavy stuff mocked, it always seemed questionable to me to see guys using some black box mocking frameworks instead of writing lightweight and smooth stubs for dependencies they wanted to abstract from the test case.
        Keep on writing those articles in a simple, approachable manner with a clear message inside.

        • http://enterprisecraftsmanship.com/ Vladimir Khorikov

          Indeed, the right tool for the right job. Thank you for your kind words! I do try to keep simplicity as the main attribute in my articles, I’m glad you noticed that!

  • Jes Slow

    Hi, where does your HandlerStub get methods such as WasCalled() and and WithStartDocument()? I’m not a Java programmer. Do you havefull code on github or something?

  • Linar Sungatullin

    Hi. I read in one book about unit tesing, that stubs are fake objects, that return some state(e.g. user name) and do nothing more. And mocks are fake objects too, but they become mocks when we verifying their behaviour in assert section.

    For example in Moq: when we create some var mock = Mock(); – it is a stub and when we call mock.Verify(…) it becomes a mock. And there is advice not to use more than one Verify(it means mock) in one test.

    What do you think about it? Anyway you articles about unit testing are awesome! 🙂

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Thanks Linar!

      Yes, I heard about such definition too. Basically, if you only fake side-effect free methods (queries) on an object, it’s a stub. Faking methods with side-effects (commands) makes a test double a mock.

      This definition makes sense as well. And it’s more abstract as it doesn’t depend on the actual implementation you choose: you can either use Moq library to implement a mock or you can create a hand-written class. As I mentioned, the classification here is a bit inconsistent, each author brings their own view on the topic.

      Regarding a single Verify() call per unit test guideline, I think it’s more related to the granularity of unit tests than to the subject of mocks versus stubs. It’s a good practice to test only one thing at a time, and multiple Verify calls is a strong sign a test doesn’t adhere to this practice.

  • Thanh Doan

    Hi Vladimir

    If I use mocking framework to create a stub, for example using Moq

    var studentRepository = new Mock();
    studentRepository .Setup(x => x.GetAll()).Returns(new List());

    Then is it ok? Do I need to create a hand-written stub object for my test.

    I still get confusing of creating Stub like example below by using Mocking framework and creating hand-written stub object.

    Many thanks,

    Thanh

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Is it the reason why I should not use Mock?

      It is. If you heavily use that stub in your tests, I’d recommend using a hand-written stub instead of relying on a mock library. However, if it’s not the case, then a mock library-generated stub would suffice.

      • Thanh Doan

        Many thanks. I will study more but I would say it is a good reason because we are now depending heavily on Moq so whenever we do refactoring then we have to update many places therefore the maintenance cost is very expensive.

      • Thanh Doan

        Hi Vladimir,

        Hope you are well.

        I might have a bad question for you. In terms of database testing, I know that we should have an integration testing for the data access codes such as the codes using LinqToSql or Dapper for i.e but what is about the Repository classes which have an abstract layer? Should we have the stub for those classes?

        The reason I asked you because at this moment, in my code base, it has been covered by the integration test that I think it is good. And in terms of unit test, those classes are mocked by Moq. However, for integration test, because we dont have a dedicated database for developer testing then it is so complicated to write test in order to keep the database consistency and clean. I got confusing about whether or not I should write stub for Repository classes? Sorry if my question is not clear.

        Also, another question, we are having a big debate about checking null for reference parameters, I always check null for reference parameters especially for constructors injection. But my colleagues seem not to be happy about that even I tried to convince them it is a very common and good practice. I came from a C++ developer so I know that checking at run time is very important for something like misuse…But I just joined the firm 6 months ago then I dont want to change their way of coding. Therefore, what do you think about checking null for reference parameter? is it necessary?

        Many thanks,

        Have a good day.

        Thanh Doan

        • http://enterprisecraftsmanship.com/ Vladimir Khorikov

          Regarding the repositories. I usually recommend writing integration tests that touch the database directly (and so there’s no need in mocking/stubbing the repositories) and unit tests that don’t involve repositories at all (and so there’s also no need in mocking/stubbing them either, but for a different reason). This will require you to have a dedicated database for each developer, though, there’s no way around this. More on this topic: http://enterprisecraftsmanship.com/2015/08/10/database-versioning-best-practices/

          Regarding checking for nulls. I do recommend checking for nulls in public methods and constructors. This way, you explicitly specify the precondition that says the input parameter should not be null. To avoid the repeating null checks, I recommend taking this approach: http://enterprisecraftsmanship.com/2015/03/13/functional-c-non-nullable-reference-types/ I also talk about this topic in depth in my other course: http://pluralsight.com/courses/csharp-applying-functional-principles

          Hope this helps,
          Vladimir

          • Thanh Doan

            Hi Vladimir,

            Thanks for your answer.

            Yeah I strongly recommend to check null for any public methods and constructors. It is also a recommended advice in Code Complete book which is at Chapter 8 and section 5 (defensive programming) but unfortunately, I am the only one in my team raised that practice but it seems to be everyone does not like it. I have no idea why. I have just joined so I don’t want to against everyone in my team, you know.

            In terms of repositories, we are having legacy code which have the following design. Imagine, we are using mixed and match between LinqToSql and Dapper :). So each Repository contains Queries and Commands classes which touch directly to database. Because we don’t have a dedicated database for each developer then writing integration test is so so difficult and complicated then in the deployment environment, we have to restore the database after every building. As a result, the test is running super slow. Therefore, at this moment, I am just trying to write integration test for Repository as it also covers Queries and Command which touch directly to database. Do you think it is ok? or do I need to write integration test for Queries and Command as well?

            One thing I dont understand your answer is that you said “unit tests that don’t involve repositories at all”. I don’t understand. Because for each repository, I always code to abstraction so if any services use repository then I can mock the repository for that kind of unit test.

            For example, I have UserRepository implement IUserRepository. UserRepository has been tested through integration test. UserRepository is used by UserService, so I can write unit test for UserService with UserRepositoryStub implementing IUserRepository or with mock of UserRepository. I think it is ok, isnt it?

            Many thanks,

            Thanh Doan.

          • http://enterprisecraftsmanship.com/ Vladimir Khorikov

            Regarding “unit tests that don’t involve repositories at all”, here I wrote more on this: http://enterprisecraftsmanship.com/2017/03/02/interfaces-for-repositories-do-or-dont/

            I wouldn’t write integration tests for Repositories specifically, only for the logic that involves those repositories. Testing technical concerns doesn’t provide a lot of value, your tests should reflect your software’s business requirements.