REST API response codes: 400 vs 500



Today, I’d like to talk about the (sometimes subtle) difference between 4xx and 5xx response codes when programming a RESTful API. I’ll try to show when to return what code (400 or 500) and introduce a simple way to implement this logic on the server side.

400 and 500 response codes from a programmer perspective

In REST, both 4xx and 5xx types of codes denote an error. The main difference between the two is whose fault that error is. A 4xx code indicates an error caused by the user, whereas 5xx codes tell the client that they did everything correctly and it’s the server itself who caused the problem. 401 (unauthorized), 403 (forbidden), 404 (not found), and so on all mean the client has to fix the problem on their side before trying again. On the contrary, 501 (not implemented), 502 (bad gateway), etc mean the request can be resent as is at a later time, when the server overcomes its current problems.

This difference is very similar to the topic of exceptions and validation errors. The guidelines here coincide almost exactly. 5xx response codes in web services are basically unexpected situations in the exceptions terminology. Essentially, all 5xx errors mean either software failures (such as bugs and configuration issues) or hardware failures (such as server outage).

Following this analogy, 4xx response codes correspond to validation errors. If the external request is incorrect for some reason, you don’t let this request through and instead return the user an error message.

Here’s a picture from my Functional C# Pluralsight course which shows this difference:

REST API response codes: 400 vs 500 - Exceptional vs non-exceptional situations

Exceptional vs non-exceptional situations

You can think of validation as of filtration. Only valid messages are allowed to come from the outside world, the rest of them are simply dropped with an explanation of what the user did wrong. There’s nothing exceptional in the act of filtering those incorrect messages out, it’s a routine operation, and thus it shouldn’t be implemented using exceptions. Otherwise, we’d face all the bad consequences of using exceptions to control the program flow. The validation is usually performed at the border of your bounded context, in application services (controllers and view models).

It’s a different situation when the message comes not from the outside world but from one of your domain classes. This could be due to one of three reasons. You either haven’t filtered an incorrect external request and let it through, or the domain class generates them on its own which means there’s a bug in it. It also could be that some external system, for example the database, returns a result you don’t know how to deal with.

All these three situations are unexpected. You can’t possibly expect your system to contain bugs. Otherwise, you would fix them in the first place. And you also expect assumptions you make about external systems to be held at all times. For instance, that your database is always online. Otherwise, they wouldn’t be assumptions.

Unexpected situations constitute a dead end for the currently executed operation. The best thing you can do if you run into one is interrupt the operation completely and stop the spread of inconsistency in your application. This is the essence of the Fail Fast principle.

How to handle 400 and 500 response codes

Alright, that’s enough of a groundwork. What about REST API response codes? There’s a direct mapping between the two concepts. All unexpected situations in your web service map to 5xx codes. All validation errors – to 4xx ones.

Sounds simple, but what does that mean to us programmers? There are several corollaries from that.

1. All unhandled exceptions should result in a 500 response code. The fact there’s an unexpected situation in your software has nothing to do with the user’s request, it’s clearly an issue on the server side, hence the 500 code. Unexpected situations show themselves up as unhandled exceptions: either bugs or hardware failures.

2. Put a single generic handler to the very top of your execution stack to handle all 500 errors. It’s difficult to deal with unexpected situations in any other way because, by definition, you can’t possibly know where they might happen. Also, return just a generic response message with the 500 code. No need to dive deep into the technical details here as the user won’t be able to make use of this information anyway.

3. Handle 400 errors separately and provide a thorough error message for each of them. Unlike 500 errors which the user can’t do anything about, 400 errors are all their fault. Therefore, you need to clearly state what that error was and how the user can avoid it in the future. The only way to accomplish it is to handle each validation error separately.

4. Never return 500 errors intentionally. The only type of errors you should be showing to the user intentionally is validation (400) errors. 500 codes are all about something you don’t anticipate to happen.

Note that while the difference between 4xx and 5xx groups of codes is important, the distinction between codes inside these groups is not. While it’s preferable to show the exact reason of why a particular request has failed (wrong URL, authentication is required, etc.), by the time a request makes its way through the framework’s layers to your code, most of those standard 404 and 401 causes are already filtered, and so you, as a programmer, can just return 400 for all remaining validation errors and 500 for server ones.

Alright, the first 3 points above are pretty self-explanatory but I think the 4th one needs elaboration. Why is that you should never return a 500 error to the user intentionally?

The explanation here is tricky. Recall that all 500 errors correspond to unexpected situations in your system: bugs and hardware failures. The moment you start doing something to handle that particular unexpected situation, it becomes an expected one. At this point, two things can happen. If it’s a bug, you simply fix it and the unexpected situation disappears completely. If it’s an incorrect user input which wrongly passed the validation, you modify the validation rules so that no such input can go through them anymore.

You can think of this process as of closing a loophole where you convert an unexpected situation into an expected one and thus eliminate that potential reason of failure.

Here’s an example:

public class CustomerController

{

    [HttpPost]

    public IHttpActionResult RegisterCustomer(string name)

    {

        var customer = new Customer(name);

        _repository.Save(customer);

        return Ok();

    }

}

 

public class Customer

{

    public string Name { get; private set; }

 

    public Customer(string name)

    {

        if (string.IsNullOrWhiteSpace(name))

            throw new ArgumentException(nameof(name));

 

        Name = name;

    }

}

This controller method registers a new customer. If we try to pass an empty name, it will throw an exception which will result in a 500 error. This error constitutes a bug: we haven’t validated the user input properly. The customer name in this example can’t be empty.

In order to fix it, we need to add validation:

public class CustomerController

{

    [HttpPost]

    public IHttpActionResult RegisterCustomer(string name)

    {

        if (string.IsNullOrWhiteSpace(name))

            return BadRequest(“Customer name can’t be empty”);

 

        var customer = new Customer(name);

        _repository.Save(customer);

        return Ok();

    }

}

Now the attempt to pass an empty customer name will result in a 400 response code with a detailed message regarding what the user did wrong. The loophole is closed.

If you need a detailed example of how to implement a generic handler for 500 errors or handle validation errors without the use of exceptions, check out the last module of the Pluralsight course I referred to earlier. Or you can go straight to its source code which I’ve put on Github.

Never mask server failures behind 400 responses

Every 500 error indicates a problem on your side. The more 500 responses your system returns, the more unstable it appears for end users. Ideally, you want to eliminate this class of errors completely. However, it’ important to note that although 5xx errors are unwanted, you shouldn’t pretend they never happen. If there’s a problem on the server side, state it clearly. It will help avoid frustration among your users and also let you track them down more easily.

On a project I participated in some years ago, there was a bug in the data access layer. The details of the bug are not important here. What important is the way the system handled that bug. Instead of returning a 500 response code (which you would normally expect from a RESTful application), that system replied with a 400 response. The code looked something like this:

[HttpPost]

public IHttpActionResult RegisterCustomer(string name, string address)

{

    try

    {

        var customer = new Customer(name, address);

        _repository.Save(customer);

 

        return Ok();

    }

    catch (Exception ex)

    {

        // All exceptions get converted into 400 errors

        return BadRequest(“Invalid request: “ + ex.Message);

    }

}

As you can see, all exceptions here get converted into 400 response codes. This is clearly a violation of the Fail Fast principle and should be avoided. In this particular case, it leads to confusion: users think it’s them who did something wrong when trying to register a customer whereas it was a bug on the server itself. Needless to say, such approach didn’t help the team to find and fix the problem as it was hidden behind a seemingly innocent validation response.

Summary

Alright, let’s summarize:

  • 4xx codes indicate errors caused by the user. They are basically validation errors.
  • 5xx codes indicate errors caused by the server itself. They are unexpected situations (bugs and hardware failures).
  • It’s important to differentiate the two. It’ll help inform users about how they can fix the problem if it appears to be on their end, and it will also help you track and fix problems on the server side. Never mask server failures behind 400 responses.
  • There are some simple guidelines regarding how to handle 400 and 500 errors.
    • All unhandled exceptions should result in a 500 response code.
    • Put a single generic handler to the very top of your execution stack to handle all 500 errors.
    • Handle all 400 errors separately and provide each of them with a thorough error message.
    • Never return 500 errors intentionally. The only way your service should respond with a 500 code is by processing an unhandled exception.

Related articles

Share




  • DAXx85

    As always fun to read! Makes me think, I should read some of your articles around this one. Kind of a nice reading to step outside of the box of my current work.

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Thanks!

  • Amir Shitrit

    Great article!
    When reporting unexpected server errors, do you mean I should return a 500 response code regardless of the type of error?

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      Thank you! Yes, that’s the preferred behavior in my opinion.

      • Amir Shitrit

        OK, but about cases where it’s important to know the exact cause if error?
        For instance if it’s a timeout, i may want to retry and if it’s a “too many requests” i may want to wait first and then retry..

        • http://enterprisecraftsmanship.com/ Vladimir Khorikov

          These two errors are actually different types of errors. Timeout is indeed a 500 one – it’s the server’s fault. Too many requests is a 400. It’s the user who caused this error and in this case the server does need to provide a thorough message explaining the problem, either by specifying an exact subcode (usually 429) or putting a description, or both.

          • Amir Shitrit

            Super.
            Thanks!

  • Piotr

    Really nice write up Vladimir,
    I have a question that is not directly related to return coders for REST, but nonetheless REST related :). I’ve developed a simple API to manage user permissions a while ago and then I saw your course on Plularsight Applying Functional Principles in C# I was really fascinated by the idea of Immutable Core and Mutable Shell and function style programming in C#.

    My main concert were unit tests which were cluttered with mock setups, mocking repository get method, save method, add, etc, mocking unit of work, mocking logger. All these seems to me like an unnecessary clutter. Once I realized I could get rid of that noise by applying immutable core/mutable shell architecture I set forth for refactoring and… I got stuck!

    How would you apply the immutable core/mutable shell approach to a REST API, given that every controller method needs to return IActionResult and unit tests should not have mocks?

    I would appreciate your help on this one 🙂

    • http://enterprisecraftsmanship.com/ Vladimir Khorikov

      That’s a great question, which is hard to answer in one comment. There’s another course of mine devoted to the very similar matter (unit testing): https://www.pluralsight.com/courses/pragmatic-unit-testing

      In short, I’d recommend extracting as much business logic as possible from controllers. This will allow you to skip covering them with unit tests because the remaining parts would be very simple and just not worth testing. It’s basically the humble object ( http://xunitpatterns.com/Humble%20Object.html ) design pattern: when you have something that is hard to test (and controllers are hard to test due to requiring the use of mocks), extract the essential part out of it and unit test that part. You can make it comply with functional principles (Immutable Core). The rest will become a simple coordination logic (Dump Shell as I call it in the Functional C# course) – you can write a few integration tests for it, but that’s it, no need to thoroughly test it as most of the domain logic is covered by unit tests.

      • Piotr

        Thanks for quick response. I saw the other course and took some ideas for testing already, good stuff :). It might be indeed too long for a comment post, but I will give it a shot.

        I followed your advice and managed to refactor to mathematical functions the commented out checks (the manager class is instantiated in the method just for testing purposes, it will be injected via ASP.NET Core dependency injection).

        UserManager with the return type are also below. The problem I have now are the _userRepository calls that are scattered across the Create method. Somehow it doesn’t feel nice, I have to call many validation methods on the UserManager that are dependent on current state (IsSuccess method). I don’t quiet grasp where the repository actions and as well logger actions should go.

        public IActionResult Create([FromBody] UserCreateModel user)
        {
        try
        {
        var man = new UserManager();
        var res = man.ValidateUser(user, ModelState);

        //if (user == null)
        // return BadRequest(ErrorMessage.NullInput($”Input {nameof(UserCreateModel)} is null”));

        //if (!ModelState.IsValid)
        //{
        // _logger.LogError(“Error while validating user {@user}”, user);
        // return BadRequest(ModelState);
        //}

        if (res.IsSuccess)
        {
        bool userExists = _userRepository.GetUsers()
        .Any(u => u.FirstName == user.FirstName && u.LastName == user.LastName);
        res = man.ValidateUserAfterExists(userExists);

        };

        var userToAdd = Mapper.Map(user);

        _userRepository.Add(userToAdd);

        if (!_userRepository.Save())
        {
        return StatusCode(500, D4GErrorMessage.DatabaseSavingError);
        }

        return res.Action;
        }
        catch (Exception exception)
        {
        _logger.LogCritical($”Failed to create usern{exception}”);
        return BadRequest(“Failed to create user”);
        }

        public class UserManager
        {
        public ControllerAction ValidateUser(UserCreateModel user, ModelStateDictionary modelState)
        {
        if (user == null)
        return new ControllerAction(new BadRequestObjectResult(ErrorMessage.NullInput($”Input {nameof(UserCreateModel)} is null”)),””, LogType.None, false);

        if (!modelState.IsValid)
        return new ControllerAction(new BadRequestObjectResult(modelState), “Error while validating user”, LogType.Error, false);

        return new ControllerAction(new CreatedAtRouteResult(“default”, user), string.Empty, LogType.None, true);
        }

        public ControllerAction ValidateUserAfterExists(bool getUsers)
        {
        if (getUsers)
        return new ControllerAction(new BadRequestObjectResult(ErrorMessage.NullInput($”Input {nameof(UserCreateModel)} is null”)), “”, LogType.None, false);
        }
        }
        public struct ControllerAction
        {
        public readonly IActionResult Action;

        public readonly bool IsSuccess;

        public readonly string LogMessage;

        public readonly LogType LogType;

        public ControllerAction(IActionResult action, string logMessage, LogType logType, bool isSuccess)
        {
        Action = action;
        LogMessage = logMessage;
        LogType = logType;
        IsSuccess = isSuccess;
        }
        }

        • http://enterprisecraftsmanship.com/ Vladimir Khorikov

          Relying on ASP.NET validators is fine but only when you are OK with coupling your domain logic with infrastructure (which is okay in many simple cases). But if you want to build an isolated domain model, I would recommend extracting the validation logic out from the ASP.NET built-in validators. This should be part of the domain model, and by making it isolated from external world (from ASP.NET), it’d become easy to unit test. Create a factory class or method that would return Result . This can be similar to what I wrote here: http://enterprisecraftsmanship.com/2015/03/07/functional-c-primitive-obsession/

          And a couple more points:
          1) Auto-mappers are good but only when you use them for mapping from domain model to DTOs, don’t use them the other way around (from UserCreateModel to User in your case).
          2) You are returning a 500 from the controller. That’s a practice I advocate against in the above post. If the repository is unable to save the entity and you can’t do anything to work around this problem, let it throw. Wrapping exceptions into returning values (the practice I advocate for in the Functional C# course) is good only for expected situations or situations that are not show-stoppers for the current transaction.