Understanding Consistency in Event Sourcing

A payments team I worked with once spent three days chasing a phantom. Their ledger showed a merchant balance of 4,812 euros in one screen and 4,798 in another, refreshed seconds apart. No bug in the arithmetic, no dropped message, nothing in the logs. The answer, once we found it, was boring and important: the two screens were reading from two different projections of the same event stream, and one was 14 events behind. Both were correct. That is the whole problem with consistency in event sourcing in a single sentence.

Event sourcing is one of the few architectural patterns I still reach for eagerly, because in a regulated payments business an append-only log of what actually happened is worth its weight in audit fees. But it drags a specific tax behind it, and the tax is confusion about what "consistent" even means once your current state is a derived thing rather than a stored thing. If you don't decide that deliberately, the system decides for you, usually at 2am during a settlement run.

Your state is a fold, not a fact

In a normal CRUD service, the balance is a row. You read it, you trust it, you move on. In event sourcing the balance doesn't exist anywhere as a stored number; it is the result of replaying MoneyReserved, PaymentCaptured, RefundIssued, and so on, from the beginning of the stream. Current state is a fold over history. That reframing is the source of every consistency question you'll face, so sit with it rather than glossing over it.

The moment you accept that state is derived, you have to ask two separate questions that CRUD lets you conflate. First: is the event log itself correct and totally ordered? Second: does the thing I'm reading reflect the latest events yet? These are genuinely different guarantees, and a huge amount of grief comes from people assuming that a durable, ordered log automatically means everything reading from it is up to date. It does not.

I like the pattern precisely because it forces this honesty. You can't pretend a read is a fact anymore. It's a snapshot of an opinion at a point in time, and you're obliged to say how stale that opinion is allowed to be.

The write side and the read side live in different worlds

Command handling and querying have almost nothing in common once you split them, and CQRS makes the split explicit. On the write side you load a stream, apply a decision, and append new events. On the read side you maintain projections tuned for whatever the UI or the reconciliation job needs. The write model can be strongly consistent within a single aggregate. The read models almost never are, and forcing them to be is where people burn entire quarters.

Here's the distinction I hammer on with every team I onboard: consistency on the write side protects your invariants, like "a merchant cannot go below their credit limit." Consistency on the read side is about freshness, like "the dashboard shows the last capture within a second." Confusing the two leads to designs that lock the whole system to protect a number that nobody was going to violate anyway.

The write side exists to prevent illegal states. The read side exists to answer questions quickly. When you ask one of them to do the other's job, you get something that is both slow and wrong.

The aggregate boundary is your consistency unit

Within one aggregate, you get strong consistency for free, and this is the part people underuse. An aggregate is a single stream with a version number, and when you append you assert the version you expected. If someone else appended in the meantime, your write is rejected and you retry against the new state. That optimistic concurrency check is the backbone of correctness in these systems, and it costs almost nothing.

The design skill is drawing the aggregate boundary so that your true invariants sit inside a single aggregate. Credit limit enforcement belongs inside the merchant account aggregate, because it's a rule about one account's own history. The temptation is always to make aggregates bigger so more rules fit inside one transaction, but big aggregates mean high contention. I once inherited a "Company" aggregate that owned every merchant, every invoice, and every payout under one roof. Under load, two payouts to two unrelated merchants would collide on the version check and one would fail. We split it and contention dropped to essentially zero.

If a rule genuinely spans multiple aggregates, that's a signal, not an inconvenience. It usually means the rule is eventually consistent in the real world too, and you should model it with a process rather than a lock.

Optimistic concurrency in practice

The mechanism is unglamorous and it works. You read the current version, make your decision, and append with that version as a precondition. The store enforces it atomically. Here's the shape of it in C#:

public async Task Capture(Guid accountId, decimal amount)
{
    var stream = await _store.Load(accountId);        // events + version
    var account = MerchantAccount.Rehydrate(stream.Events);

    if (account.Available < amount)
        throw new InsufficientFundsException(accountId);

    var newEvent = new PaymentCaptured(accountId, amount, DateTime.UtcNow);

    // Fails if another writer moved the version since we loaded
    await _store.Append(accountId, expectedVersion: stream.Version, newEvent);
}

The line that matters is the expectedVersion argument. If two capture requests race, one appends at version 42 and the other is told "sorry, we're at 43 now," so it reloads, re-checks the balance, and either succeeds or fails honestly. No distributed lock, no advisory lock in the database, no coordination service. On one system this single pattern let us handle capture bursts of a few thousand per minute on modest hardware without a single double-spend, and the retry rate stayed under one percent.

Read models are eventually consistent, and that is fine

Projections lag. A projector consumes events and updates a read table, and there is always a gap between the append and the update. Usually it's milliseconds. During a redeploy or a backlog it can be seconds, occasionally minutes if you've done something silly. The engineering question is not "how do I make the lag zero" but "how much lag can each reader tolerate, and what do I do at the edges of that tolerance."

Different readers want wildly different answers here:

• A merchant dashboard is happy with a second or two of staleness and nobody notices.
• A fraud rule evaluating the same payment probably wants to read from the write model directly, not a projection, because being 500ms behind can mean approving something it should have blocked.
• An end-of-day reconciliation report wants a specific, frozen point in the stream, not "now," so it should read up to a known event position and stop.
• A "you just clicked submit, show me the result" screen is the classic trap, because the user expects to see their own write immediately.

That last one is read-your-own-writes, and it's where naive eventual consistency shows its seams. The user captures a payment, the UI redirects to a page backed by a projection, the projection hasn't caught up, and the payment appears to have vanished. Users file support tickets about money vanishing. You do not want those tickets.

Closing the gap when you actually need to

There are a handful of honest techniques for the read-your-own-writes problem, and I've shipped all of them at various times. The write handler can return the new stream version to the client, and the read request can then say "wait until the projection has caught up to at least version N before answering." That turns a vague race into a bounded wait, usually a few tens of milliseconds, with a timeout that falls back to reading the write model.

Alternatively, for the one screen that must be exact, just query the aggregate directly and rehydrate it. It's a few extra events to fold and it's completely current. I reserve this for genuinely sensitive views. Do it everywhere and you've quietly rebuilt a synchronous system, throwing away the scaling benefit that made you choose projections in the first place. The discipline is spending strong consistency only where a human or a regulation actually demands it.

When a rule spans aggregates, use a process

Some rules cannot fit in one aggregate. Moving funds between two merchant accounts touches two streams, and you cannot append to both in one atomic transaction without recreating the very coupling event sourcing helped you avoid. The right answer is a process manager, sometimes called a saga: it listens for FundsDebited on account A, then issues a command to credit account B, and it owns the job of making sure the whole thing either completes or compensates.

This is eventual consistency by design, and it mirrors how money actually moves. A card capture and its settlement are not simultaneous in the real world; there's a delay of a day or more, and the system is temporarily "inconsistent" in the sense that the money is neither fully here nor fully there. Modeling that with a process rather than pretending it's instantaneous is more honest and far more resilient. When the credit fails, you have an explicit place to issue a compensating event and an explicit state that says "in flight," which auditors love and on-call engineers love more.

Snapshots and the cost of the fold

Replaying ten million events to answer one query is a bad time. Snapshots are the pragmatic fix: periodically store the folded state at a known version, then on load you start from the snapshot and replay only the events since. It's a cache, and like every cache it's a correctness risk if you treat it as a source of truth. The events remain authoritative; a snapshot is only ever an optimization you can throw away and rebuild.

The trap I've watched teams fall into is writing business logic that reads the snapshot and trusts it blindly, then changing the fold logic later. Now old snapshots encode the old rules and new events encode the new ones, and your state is a Frankenstein of both. Version your snapshots, and be willing to invalidate every one of them and replay from zero when the fold changes. If a full replay is too slow to contemplate, that's telling you your streams are too long or your snapshot cadence is wrong, not that you should skip the replay.

Conclusion

The mental shift that makes all of this click is to stop asking "is my system consistent?" and start asking "consistent with respect to what, for whom, and by when?" Consistency in event sourcing isn't a switch you flip on. It's a budget you allocate: strong where invariants and regulators demand it, eventual everywhere else, and made explicit at every boundary so no one is surprised. The teams that struggle are the ones who wanted a single global answer. The teams that thrive treat freshness as a per-reader service level, write it down, and defend it.

If you take one thing from those three days of chasing a phantom balance, let it be this: the two screens weren't broken, our expectations were. The bug was that nobody had decided how stale each view was allowed to be. Decide that on purpose, up front, and event sourcing stops feeling like a source of mysteries and starts feeling like what it is, which is the most honest record of your business you'll ever build.