Skip to content

step 18 / 32 · 6 min

When the answer depends on who you ask

One database on one machine has one version of the truth. Add a replica, a cache or a second service and that stops being true, and most of the difficulty in distributed systems follows.

Strong and eventual

Strong consistency: every read returns the most recent write. Simple to reason about, and it requires coordination, which costs latency and availability.

Eventual consistency: reads may be stale, and converge given time. Cheap and fast, and the application must tolerate seeing old data.

Neither is better. A view counter can be eventually consistent. An account balance cannot.

Read your own writes

The specific case that produces bug reports:

code
user updates their profile   -> written to the primary
page reloads                 -> read from a replica
the replica is 200ms behind  -> they see the old value

The user did everything right and the system appears to have lost their work.

Fixes, in order of preference: read from the primary for a short window after a write by that user, or route that user's reads to the primary, or wait for the replica to catch up before responding.

This guarantee has a name, read your own writes, and it is usually the only one users actually notice the absence of.

CAP, stated narrowly

During a network partition, a system must choose between refusing requests and serving possibly stale data.

That is all it says. It is not a claim about normal operation, and it does not say you must abandon consistency in general. It is quoted far more broadly than it holds.

Transactions across services

A monolith wraps two changes in one transaction. Two services cannot.

Two phase commit coordinates them and blocks if the coordinator fails, which is why it is rarely used at scale.

Saga: a sequence of local transactions, each with a compensating action if a later step fails.

code
reserve stock  -> charge card  -> confirm order
                      fails
release stock  <- compensate

The system is briefly inconsistent, and compensation is not the same as a rollback: a charge that was refunded still happened and is visible on a statement. Designing what the user sees during that window is part of the work.

Idempotency again

Retries and at least once delivery mean an operation may run twice. A key generated once by the caller and recognised by the server is what makes that safe, and it is the single most valuable habit in distributed work.