Your Users Shouldn't Have to Wait: Learn Message Queues

This is Part 10 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear.

In Part 9, we solved the problem of data that had grown too large for a single database. We split it across multiple shards, each holding its piece of the whole, so that no single machine ever had to carry everything. At that point, the architecture could scale in almost every direction we'd tried to push it. Traffic was distributed across application servers. Repeated database work was absorbed by the cache. Read traffic was spread across replicas. Data itself was partitioned across shards.

And yet. We ended Part 9 by noticing something that none of those solutions addressed. Some user requests trigger a lot of downstream work. Saving an order is one thing. But saving the order, sending a confirmation email, generating an invoice, updating inventory, firing off a notification, recording an analytics event, triggering the recommendation engine: that's an entirely different conversation. Right now, all of that happens before the user gets a response. The question we left with was this: what if they didn't have to wait for all of it?

Section 1: The User Doesn't Need Everything Right Now

Before we look at any solution, it's worth asking a simpler question. When a user places an order, what do they actually need to know before they can move on?

They need to know the order was received. They need confirmation that the important thing happened: their money was accepted, their items are reserved, the transaction is real. That's it. That's what they're waiting for.

They do not need to wait for the confirmation email to land in their inbox. They do not need to wait for the invoice to be generated and stored somewhere. They do not need to wait for the analytics system to record that this purchase happened. They certainly don't need to wait for the recommendation engine to update its model based on what they just bought.

All of that will happen. It should happen. But none of it needs to happen before the user gets their confirmation screen.

This seems obvious when you say it directly. Of course the user doesn't need to wait for the analytics event. Of course they don't need to hold their breath while the recommendation system recalculates. But the way most applications are initially built, that's exactly what happens.

Section 2: Doing Everything Synchronously

Here's what a typical order flow looks like when everything is wired together the simple, obvious way.

User clicks "Place Order"
       |
       v
Application receives request
       |
       v
Save order to database (50ms)
       |
       v
Send confirmation email (200ms)
       |
       v
Generate invoice (120ms)
       |
       v
Update inventory (80ms)
       |
       v
Record analytics event (90ms)
       |
       v
Trigger recommendations (150ms)
       |
       v
Return response to user

Total wait: ~690ms

The user clicked a button and waited almost 700 milliseconds for a confirmation screen. More than half a second, for a response to something they did in an instant. And that's assuming nothing goes wrong.

What if the email service is having a slow moment and takes two seconds instead of 200 milliseconds? The user waits two seconds. What if the analytics system is down entirely? The request fails, and the user gets an error, even though the order itself was saved perfectly.

The application has tied its response time and its reliability to every piece of downstream work it performs. Every slow step makes the user wait longer. Every failing step makes the whole request fail.

That's not a hardware problem. It's not a database problem. It's an architectural problem. The application is doing work in a sequence when most of that work doesn't actually depend on the steps before it. The confirmation email doesn't need the invoice to be generated before it can be sent. The analytics event doesn't need the email to succeed before it can be recorded. These tasks are all independent of each other. They're only sequential because that's how they got wired together.

So the question becomes: what would it look like to stop treating them as sequential?

Section 3: Put the Work in a Queue

The insight is simple once you see it. Not all work needs to happen now. Some work needs to happen eventually, but the user doesn't need to wait for it.

If that's true, then instead of doing all that work before responding, the application can do this:

• Save the order. That's the important part, and it must happen now.
• Make a note that a confirmation email needs to be sent.
• Make a note that an invoice needs to be generated.
• Make a note that an analytics event needs to be recorded.
• Return a response to the user.

Then, separately, something else comes along, reads all those notes, and does the work.

User clicks "Place Order"
       |
       v
Application receives request
       |
       v
Save order to database (50ms)
       |
       v
Leave notes for background work
       |
       v
Return response to user

Total wait: ~60ms

The user gets their confirmation in 60 milliseconds instead of 690. Then, in the background, the email goes out, the invoice is generated, the analytics are recorded. The user never waits for any of it.

This is the idea behind a message queue. Instead of doing background work inline, the application puts a message into a queue. Each message is a piece of work that needs to happen: "send this email," "generate this invoice," "record this event." The queue holds onto those messages. Something else picks them up and does the actual work.

The application that creates messages is called the producer. It produces work to be done. The queue is the place that holds work waiting to be processed. It doesn't do the work itself. It holds the work. The thing that picks up messages and processes them is called a consumer or worker. It consumes messages from the queue and does the actual task.

BEFORE:
User --> Application --> [save] --> [email] --> [invoice] --> [analytics] --> Response

AFTER:
User --> Application --> [save] --> Queue --> Response
                                     |
                      .--------------+--------------.
                      |              |              |
                 [Worker]        [Worker]        [Worker]
                   email          invoice        analytics

The user's request touches the queue for a moment, hands off the work, and returns a response. The workers operate completely independently. They process messages at their own pace, without the user having to wait.

Section 4: Let Workers Do the Work Later

This separation between "the work that happens now" and "the work that happens later" is called asynchronous processing. The user's request doesn't wait for every piece of work to complete before finishing. It hands off what it can, responds quickly, and trusts that the