Integration testing or how to sleep well at nights

By Vladimir Khorikov

Unit testing is good at checking the correctness of your code in isolation, but it’s not a panacea. Only integration tests can give us confidence that the application we develop actually works as a whole. They are also a good substitute for mocks in the cases where you can’t test important business logic without involving external dependencies.

Integration testing

Integration tests operate on a higher level of abstraction than unit tests. The main difference between integration and unit testing is that integration tests actually affect external dependencies.

The dependencies integration tests work with can be broken into two types: the ones that are under your control, and the ones that you don’t have control over.

Database and file system fall into the first category: you can programmatically change their internal state which makes them perfectly suitable for integration testing.

The second type represents such dependencies as email SMTP server or enterprise service bus (ESB). In most cases, you can’t just wipe out the side effects introduced by invoking an email gateway, so you still need to somehow fake these dependencies even with integration tests. However, you don’t need mocks to do that. We’ll discuss this topic in a minute.

It’s almost always a good idea to employ both unit and integration testing. The reason is that, with unit tests, you can’t be sure that different parts of your system actually work with each other correctly. Also, it’s hard to unit test business logic that don’t belong to domain objects without introducing mocks.

A single integration test cross cuts several layers of your code base at once resulting in a better return of investments per line of the test code. At the same time, integration testing is not a substitution for unit testing because they don’t provide as high granularity as unit tests do. You can’t just cover all possible edge cases with them because it would lead to significant code duplication.

A reasonable approach here is the following:

  • Employ unit testing to verify all possible cases in your domain model.
  • With integration tests, check only a single happy path per application service method. Also, if there are any edge cases which cannot be covered with unit tests, check them as well.

Integration testing example

Alright, let’s look at a concrete example of how we can apply integration testing. Below is a slightly modified version of the method from the previous post:

public HttpResponseMessage CreateCustomer(string name, string email, string city)

{

    Customer existingCustomer = _repository.GetByEmail(email);

    if (existingCustomer != null)

        return Error(“Customer with such email address already exists”);

 

    Customer customer = new Customer(name, city);

    _repository.Save(customer);

 

    if (city == “New York”)

    {

        _emailGateway.SendSpecialGreetings(customer);

    }

    else

    {

        _emailGateway.SendRegularGreetings(customer);

    }

 

    return Ok();

}

How can integration tests help us in this situation?

First of all, they can verify that the customer was in fact saved in the database. Secondly, there’s an important business rule here: customers’ emails must be unique. Integration testing can help us with that as well. Furthermore, we send different types of greeting emails depending on the city the customer’s in. That is also worth checking.

Let’s start off with testing a happy path:

[Fact]

public void Create_customer_action_creates_customer()

{

    var emailGateway = new FakeEmailGateway();

    var controller = new CustomerController(new CustomerRepository(), emailGateway);

 

    controller.CreateCustomer(“John Doe”, “john@doe.com”, “Some city”);

 

    using (var db = new DB())

    {

        Customer customerFromDb = db.GetCustomer(“john@doe.com”);

        customerFromDb.ShouldExist()

            .WithName(“John Doe”)

            .WithEmail(“john@doe.com”)

            .WithCity(“Some city”)

            .WithState(CustomerState.Pending);

 

        emailGateway

            .ShouldSendNumberOfEmails(1)

            .WithEmail(“john@doe.com”, “Hello regular customer!”);

    }

}

Note that we pass a real customer repository instance to the controller and a fake email gateway. Here, the repository is a dependency we have control over, whereas email gateway is the dependency we need to fake.

Also note the DB class and the heavy use of extension methods, such as ShouldExist, WithName and so on. The DB class is a utility class that helps gather all test-specific database interactions in a single place, and the extension methods allow us to check the customer’s properties in a narrow and readable way.

The test also verifies that an appropriate email was sent to the newly created customer. In this particular case, the email should be sent with “Hello regular customer” subject. We’ll look at the fake email gateway closer shortly.

Here’s another test:

[Fact]

public void Cannot_create_customer_with_duplicated_email()

{

    CreateCustomer(“john@doe.com”);

    var controller = new CustomerController(

        new CustomerRepository(),

        new FakeEmailGateway());

 

    HttpResponseMessage response = controller.CreateCustomer(“John”, “john@doe.com”, “LA”);

 

    response.ShouldBeError(“Customer with such email address already exists”);

}

Here we verify that no two customers can have the same email.

The two tests shown above allow us to make sure that all three layers (controllers, the domain model, and the database) work together correctly. They immediately let us know if there’s anything wrong with the database structure, object-relational mappings, or SQL queries, and thus give us a true confidence our application works as a whole.

Of course, it’s not 100% guarantee because there still could be issues with ASP.NET Web API routing or SMTP server settings. But I would say that integration testing, in conjunction with unit testing, provide us about 80% assurance possible.

Alright, and finally here’s the third integration test which verifies that New Yorkers receive a special greetings letter:

[Fact]

public void Customers_from_New_York_get_a_special_greetings_letter()

{

    var emailGateway = new FakeEmailGateway();

    var controller = new CustomerController(new CustomerRepository(), emailGateway);

 

    controller.CreateCustomer(“John”, “john@doe.com”, “New York”);

 

    emailGateway

        .ShouldSendNumberOfEmails(1)

        .WithEmail(“john@doe.com”, “Hello special customer!”);

}

Implementing stubs for external dependencies

Now, let’s take a closer look at the fake email gateway we used in the tests above:

public class FakeEmailGateway : IEmailGateway

{

    private readonly List<SentEmail> _sentEmails;

    public IReadOnlyList<SentEmail> SentEmails

    {

        get { return _sentEmails.ToList(); }

    }

 

    public FakeEmailGateway()

    {

        _sentEmails = new List<SentEmail>();

    }

 

    public void SendRegularGreetings(Customer customer)

    {

        _sentEmails.Add(new SentEmail(customer.Email, “Hello regular customer!”));

    }

 

    public void SendSpecialGreetings(Customer customer)

    {

        _sentEmails.Add(new SentEmail(customer.Email, “Hello special customer!”));

    }

 

    public class SentEmail

    {

        public string To { get; private set; }

        public string Subject { get; private set; }

 

        public SentEmail(string to, string subject)

        {

            To = to;

            Subject = subject;

        }

    }

}

As you can see, it just records recipients and subjects for all emails sent during its lifespan. We then use this information in our integration tests.

This implementation has a significant flaw which makes it no better solution than the mocks I argued against in the first article: it’s too brittle as it mimics every single method in the IEmailGateway interface. This stub would either stop compiling or report false positive every time you make a change in the interface.

Also, it makes a heavy use of the code duplication. Note that the letters’ subjects are just typed in directly resulting in virtually no advantage over mocks.

Integration testing: One-to-one interface implementation

One-to-one interface implementation

A much better approach would be to extract a separate class – an email provider – with a single method to which the email gateway would resort in order to send an email. This provider then can be substituted by a stub:

Integration testing: extracting a provider

Extracting a provider

Unlike the previous version, this stub is stable as it doesn’t depend on the actual emails sent by the email gateway. It also doesn’t introduce any code duplication – it just records all incoming emails as is:

public class FakeEmailProvider : IEmailProvider

{

    public void Send(string to, string subject, string body)

    {

        _sentEmails.Add(new SentEmail(to, subject));

    }

 

    /* Other members */

}

Summary

Integration testing is a good alternative to unit testing with mocks because of two reasons:

  • Unlike unit tests with mocks, integration tests do verify that your code works with external dependencies correctly.
  • For the dependencies you don’t have control over, integration testing makes use of stubs instead of mocks. In most cases, stubs are less brittle and thus more maintainable choice. This point correlates to an old discussion about behavior vs state verification (also known as the classicist and the mockist schools). We’ll return to this topic in the following articles.

In the next post, we’ll talk about the most important TDD rule.

If you enjoyed this article, be sure to check out my Pragmatic Unit Testing Pluralsight course too.

Other articles in the series

LinkedInRedditTumblrBufferPocketShare




  • Peter Schulz

    Thank you for this article.

    It does hit a very important point on “limited” unit-testing. I believe that, in some developer’s minds, the procedure what you call integration is mixed up with unit testing, i.e. they have done integration testing while believing this to be unit-testing …

    I would see following “hybrid” approach is the most natural, and follows from the agile principles. This would apply to a multi-tier application

    * You define your model on a very basic level and unit test just that.
    (that automatically includes database testing if you are using an ORM tool)
    * Create client code. Unit test only its internal workings, not the connectivity to the
    server model tier.
    * Now comes the step you call integration testing – you test the functionality of the
    client by actually having the server run. No mock – the actual server process runs.
    Since you have unit-tested the server methods already in the first step, there is
    little to fail apart from the “working-together” of these components.

    An interesting point is that when you do not use an ORM tool in the first step, then even testing the model methods becomes automatically “integration testing”, i.e. you test that the database works as you desire in accordance with the model tier code. In such a case, you actually never “unit-test” at all … it’s all integration testing of sorts.

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Interesting points, thanks for sharing!

  • http://rpajak.com/ Pellared

    Thank you for this nice article!

    I would point one big advantage of using stubs, fakes and spies (in your case the FakeEmailProvider is rather a spy than a fake): it is easier to find the design problems of code under test in the test code. Because for example if some interface is bad (for example to big) than making a test double is hard. It might be worth to mention that the FakeEmailProvider could be easily implemented using some mocking framework, however the flexibility and simplicity of the handwritten class is really powerful. I really like the tests you proposed – moreover then one can use FluentAssertions – a library that I truly love and generally has more power than the mocking “verifications”.

    Just one more problem: how do you solve testing if the sequence of the “interaction” is correct. Given your example, suppose that it is really important that you first store the customer in the repository and then send the email. How would you test it in a “clean” way?

    Looking forward to reading your reply and next article!

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      >”I would point one big advantage of using stubs, fakes and spies”
      100% agree! And I too use FluentAssertions, really powerful and helpful tool!

      Regarding the operation sequence testing – I usually don’t do that. The importance of the order in which operations are executed means that there should be some side-effect or an edge case that depends on that order. In this case, I just compose tests in a way that allows me to test all of those side-effects. In most cases, it is not the operation sequence that important, it’s the side effects this sequence leaves when ran under different circumstances, so it’s a good idea to approach the testing of them from the client point of view, without knowing inner implementation details.

      You actually touched upon an important topic, which I’m going to write about in the next post. So important that I even decided to name it “The most important TDD rule” :)

      Thank you for your comment!

  • Idahosa Edokpayi

    I am a SharePoint guy who has struggled with how to get SharePoint to testability and this kind of thinking is probably what’s necessary. You are expressing in writing some things I have felt for a while but never formally expressed anywhere. Thanks so much!
    http://whoisidaho.com

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Thanks for your kind comment!