Two ways to stop doing work now

Both make a system faster by not doing something during the request. Both introduce a second source of truth or a delay, and that is the cost.

Caching

Store a computed result so the next request does not recompute it.

request -> cache hit?  -> yes: return it
                       -> no:  compute, store, return

The layers, each with a different reach:

In memory caching per instance is the one that surprises people: with four instances there are four caches, so two users can see different data depending on which one served them.

What to cache

Read far more often than it changes, and slight staleness is acceptable. A product catalogue, yes. An account balance, no.

Two failure modes to know by name:

Stampede. A popular entry expires and a thousand requests recompute it at once, hitting the database harder than having no cache. Fixes: a lock so one request recomputes, or refreshing before expiry.

Cold cache. After a deployment or restart, everything misses. A system that only survives because of its cache falls over at exactly the moment it restarts.

Queues

Accept the work, do it later.

request -> put a message on the queue -> respond immediately
                                      -> a worker processes it

Good for anything the user does not need the result of: sending email, generating reports, resizing images, syncing to another system.

What it buys: fast responses, absorbing spikes, and surviving a dependency being down because the work waits rather than failing.

What it costs:

• the work happens later, so anything depending on it must tolerate that
• failures are invisible to the user, so they need alerting or they go unnoticed
• a message may be delivered twice, so the work should be safe to repeat
• there is now a queue to operate, which can back up

Idempotency is the requirement people underestimate. Most queues guarantee at least once delivery, so charging a card from a queue consumer without a key will eventually charge twice.

A dead letter queue

Messages that fail repeatedly go to a separate queue rather than retrying forever. Without one, a single poison message can block a consumer indefinitely, and nothing reports it.