How to Version an Event Schema

The first time an event schema change took down a downstream service, I was three years into running payments infrastructure and I still thought of events as fire-and-forget. We added a currency field to a PaymentSettled event, shipped it on a Tuesday, and by Wednesday morning the reconciliation team was staring at a consumer that had silently stopped committing offsets because a strict deserializer rejected the extra property. No error page, no alert. Just a queue quietly backing up while the money kept moving.

That incident taught me something I now repeat to every engineer who joins my team: an event is a contract, and the moment another team reads it, you no longer own it alone. Versioning is not a nice-to-have you bolt on later. It is the difference between a schema you can change safely and one you are afraid to touch.

Events Are Forever, and You Should Plan For That

The mental trap is treating an event like an API response. An API call is a conversation that ends when the caller hangs up. An event is different. Once you publish it to a durable log like Kafka, that record can sit there for weeks, and someone might replay it into a brand-new consumer that did not exist when you wrote it. You are not just serving today's readers. You are serving every reader who will ever process that offset, including the ones your company has not hired yet.

This changes the economics of every field you add. In a request-response world, a field you regret can be deprecated and removed within a release cycle. In an event world, that field is baked into history. If you shipped a poorly named amount field that mixed cents and dollars for two months in 2024, that ambiguity lives in your topic forever, and every consumer replaying that range has to know about it. I have come to treat every event property as a permanent public commitment, because in practice that is exactly what it is.

Get Precise About Backward and Forward Compatibility

Most schema arguments go in circles because people use "compatible" to mean different things. There are two directions and they are not the same, so it pays to be exact about which one you are defending.

• Backward compatible: a new consumer can read old events. You add a field, and code written for v2 still makes sense of a v1 record it finds in the log.
• Forward compatible: an old consumer can read new events. You add a field, and the v1 consumer that has not been redeployed simply ignores what it does not recognize.
• Full compatibility: both hold at once. This is the target for anything with more than one consumer team, and it is stricter than most people assume.

The failure I described at the top was a forward-compatibility break. The producer moved forward; the consumer had not. Forward compatibility is the one teams forget, because during local testing the producer and consumer are always on the same version. You only feel the pain in production, where deploys are staggered and a consumer might lag the producer by days. My rule of thumb: if you cannot deploy producer and consumer in either order without breaking, you do not have a versioning strategy, you have a coordination problem you are pretending is solved.

Make Additive Changes the Default, Removal the Exception

Ninety percent of the schema changes I have seen are additive. You want to attach a new piece of data to an event that already exists. The good news is that additive changes are the safe ones, provided you follow two disciplines. New fields must be optional, and consumers must ignore fields they do not recognize. Get both of those right and you can add fields for years without a single coordinated deploy.

The second discipline is the one people skip. Enforcing "ignore unknown fields" is a consumer-side setting, and it has to be deliberate. Defaults vary wildly across serializers. System.Text.Json skips unknown members, Avro in strict mode rejects them, and the outage at the top of this post was a strict deserializer doing exactly what it was told. Configure the behavior explicitly. Do not inherit it by accident.

var options = new JsonSerializerOptions
{
    // Old consumers must tolerate fields added by newer producers.
    UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip,
    PropertyNameCaseInsensitive = true
};

// A v1 consumer reading a v2 event: the new "settlementCurrency"
// field is present in the JSON but absent from the v1 record type.
// With Skip, this deserializes cleanly instead of throwing.
public record PaymentSettledV1(
    Guid PaymentId,
    long AmountMinorUnits,
    DateTimeOffset SettledAt);

var evt = JsonSerializer.Deserialize<PaymentSettledV1>(payload, options);

That one setting would have prevented my Tuesday outage. Removing a field is where discipline really matters, because removal is a backward-incompatible act dressed up as cleanup. You never remove a field the moment you stop populating it. You stop populating it, wait until every consumer has moved off it, confirm that with real telemetry rather than a Slack thread, and only then delete it from the schema. That waiting period is measured in months for anything important, and I have learned not to rush it.

Put a Version on the Event, Not Just in Your Head

Even with additive discipline, you will eventually need a real version. Renaming a field, splitting one event into two, changing the meaning of a value: these are breaking changes, and pretending otherwise just moves the pain downstream. When that day comes, you want an explicit version marker already in place so consumers can branch cleanly.

I put the version in two places. There is a schemaVersion field inside the payload, and there is a version identifier in the message metadata or headers so a consumer can route without deserializing the whole body. The event type name itself carries a major version too, so PaymentSettled and PaymentSettled.v2 are distinct topics or distinct type tags. Redundant? A little. But when you are debugging a poison message at 2am, having the version visible in three places you can grep is a gift you give your future self.

A schema without a version is a schema you are afraid to change. The version number is not bureaucracy. It is the permission slip that lets you evolve the event without a war-room deploy.

Use a Schema Registry and Enforce Compatibility in CI

Human discipline does not scale past about two teams. The moment you have five services publishing to a shared bus, someone will ship a breaking change during a crunch, and they will do it without malice. A schema registry that rejects incompatible changes at build time is how you turn a social norm into an engineering guarantee.

We run Confluent Schema Registry with compatibility set to FULL_TRANSITIVE on our payment topics, which means a proposed schema is checked against every historical version, not just the immediately previous one. That transitive part matters more than people expect. I have watched a change pass against version 4, only to break a consumer still reading version 2 during a long replay. The registry check is a gate in CI, not a runbook step, so a breaking change fails the pull request before a human ever has to catch it. The best control is the one nobody has to remember.

Prefer Upcasting Over Forcing Everyone to the Latest Version

When you do introduce a breaking v2, you have a choice about who absorbs the cost. The lazy option is to demand every consumer upgrade before you ship. That turns one team's improvement into an org-wide migration and it never actually finishes. The better pattern, which I borrowed from the event-sourcing world, is upcasting: you transform old events into the new shape at read time, in one place, so most consumers never learn that v1 existed.

Concretely, you write a small function that takes a v1 event and produces a v2, filling in sensible defaults for anything the old version lacked. That function lives in a shared library or in the consumer's deserialization boundary. New code only ever sees v2. Old events in the log stay untouched, which they must, because you cannot rewrite history in an append-only store anyway. The cost is that you carry that upcaster forever, or at least until you snapshot and truly retire the old range, which for us has meant keeping a handful of them around for years. That is a fair trade for not blocking six teams on a rename.

Name Fields for the Reader Who Has No Context

Half the versioning pain I have caused myself was really a naming problem in disguise. A field called amount forces a v2 the day you need to know whether it is gross or net, minor units or major, before or after fees. A field called grossAmountMinorUnits never needs that clarification because the ambiguity was designed out on day one. Good names are the cheapest form of versioning, because the version you never have to create costs nothing.

My hard-won conventions are boring on purpose. Money is always in minor units as an integer, never a float, and the currency is always adjacent to it. Timestamps are always UTC with an offset and always named with an -At suffix. Enumerations get string values, not ordinals, because a numeric enum that gets reordered is a silent data-corruption bug waiting for a replay. None of this is clever. It is just the accumulated scar tissue of every field I have had to version because I named it lazily the first time.

Test the Compatibility, Do Not Assume It

The gap between "should be compatible" and "is compatible" is where outages live. I keep a set of golden event files, real serialized payloads from each historical version, checked into the repo. Every build deserializes all of them against the current consumer code. If a change breaks the ability to read a 2023 event, the test goes red, and it goes red on the laptop of the person who made the change rather than in production during a replay.

This is cheap insurance and it catches the subtle regressions that a schema registry alone will miss, like a consumer that technically parses an old event but then throws in business logic because it assumed a field would be present. Compatibility is not just "does it deserialize". It is "does the code do something sane with the result". Those golden files are the only thing I trust to answer that second question honestly.

Conclusion

If I could hand my earlier self one sentence, it would be this: design events as if you will never be allowed to delete a field, because functionally you will not. That single constraint pushes you toward optional additive changes, explicit versions, careful names, and automated compatibility gates, not because a process document told you to, but because they are the only moves that stay cheap over years. The teams that treat event schemas casually do not feel it in the first month. They feel it in year two, when the topic has forty fields, three of them lie about their own meaning, and nobody dares touch the producer. Put the version in now, wire the registry check into CI this week, and write the boring field names today. The quiet queue backing up on a Wednesday morning is a memory I would rather you skip.