Everything you depend on will fail

A payment provider has a bad ten minutes. Your checkout calls it with no timeout, so each request waits. Threads fill up. The whole site stops responding, including pages that never touch payments.

One dependency had a bad ten minutes and took everything down with it.

Timeouts

Every call over a network needs one. Without it, "slow" becomes "hung", and hung requests accumulate until there is nothing left to serve with.

And they must decrease as you go deeper:

browser        10s
  api           8s
    service     6s
      database  4s

If your caller waits 30 seconds and you wait 60 on the thing behind you, you are doing work for a result nobody will receive, while holding a connection and a thread to do it.

Retries, carefully

Retrying helps with a blip and makes an outage worse. If every client retries immediately, a struggling service receives several times its normal traffic at the exact moment it has least to spare.

Three parts to doing it properly:

• Backoff: wait longer each time.
• Jitter: randomise the wait, or every client returns together and the spike simply moves.
• A circuit breaker: after enough failures, stop calling for a while and fail fast. This is what stops you holding a struggling dependency under.

Retrying safely needs idempotency

A timeout tells you nothing. The request may have been processed and only the response lost. Retrying blindly charges the customer twice.

The fix is an idempotency key: the client sends a unique value per logical operation, the server records it with the result, and a repeat returns the stored result instead of doing the work again.

Degrade rather than fail

If recommendations are down, show the page without them. If search is down, show categories. A page that loses a panel is better than a page that loses itself, and deciding in advance which parts are optional is most of the work.