Building Reliable Event-Driven Systems: Event Schemas, Versioning, Contract Testing and Events vs Commands (part-3)

Event Schema Evolution with Event Versioning

Event Schemas Will Eventually Change

No event schema stays the same forever. As businesses grow, regulations shift, products gain new features, and processes become more complex, the data shared between services must evolve as well. This evolution is not optional-it is a natural consequence of a system adapting to changing requirements.

Many teams initially assume they can simply update an event whenever needed. This assumption may hold when there is only one producer and one consumer, but real-world systems rarely remain that simple. Over time, multiple consumers emerge, each with its own responsibilities and release cycles. A typical system often looks like this:

OrderConfirmed
    |
    +------------------+-------------------+
    |                  |                   |
    v                  v                   v
Inventory          Billing           Notification
                                        |
                                        v
                                   Analytics
                                        |
                                        v
                               Customer Insights

Each consumer evolves independently. Some services may deploy updates weekly, while others might release changes quarterly. In some cases, consumers may even belong to external teams with entirely different priorities and timelines. Because of this, producers cannot assume that all consumers will upgrade simultaneously.

Schema evolution, therefore, is not just about modifying data structures. It is fundamentally about maintaining compatibility across independently evolving systems.

Compatibility Is More Important Than Version Numbers

When discussing schema evolution, teams often focus immediately on versioning. While versioning is useful, compatibility is far more critical. Without compatibility, versioning alone cannot prevent system breakage.

Consider the following event:

{
  "orderId": "ORD-1001",
  "customerId": "CUS-501",
  "totalAmount": 249.99
}

Now imagine a new requirement introduces currency. One approach might replace the existing field entirely:

{
  "orderId": "ORD-1001",
  "customerId": "CUS-501",
  "amount": {
    "value": 249.99,
    "currency": "USD"
  }
}

Although the data model has improved, this change breaks every existing consumer that depends on the original structure. The producer has evolved, but the contract has not been preserved. The goal of schema evolution is to allow both producers and consumers to evolve independently without causing disruptions.

Backward and Forward Compatibility

Compatibility is often described using formal definitions, but a practical understanding is more useful when designing real systems.

Backward Compatibility

Backward compatibility ensures that existing consumers continue to function when producers introduce newer versions of events. This is one of the most important principles in event-driven systems.

Consider Version 1 of an event:

{
  "orderId": "ORD-1001",
  "customerId": "CUS-501"
}

Now Version 2 introduces an additional field:

{
  "orderId": "ORD-1001",
  "customerId": "CUS-501",
  "currency": "USD"
}

Older consumers can safely ignore the new field, allowing them to continue operating without modification. This makes adding optional fields one of the safest ways to evolve a schema. In contrast, removing fields is far more risky because it can break existing consumers.

Forward Compatibility

Forward compatibility addresses the opposite scenario, where newer consumers must handle older events produced by systems that have not yet been upgraded.

For example, a new consumer might expect:

{
  "orderId": "...",
  "customerId": "...",
  "currency": "USD"
}

However, older producers may still emit:

{
  "orderId": "...",
  "customerId": "..."
}

In this case, consumers must be designed to handle missing fields gracefully. They might use default values, leave fields empty, or apply fallback logic. This approach ensures that consumers remain resilient even when the system evolves asynchronously. Consumers should never assume that every field will always be present, as distributed systems rarely evolve in perfect synchronization.

Compatibility Is a Team Discipline

Most compatibility issues arise not from technical limitations but from incorrect assumptions. A common example is the belief that all consumers have already upgraded to the latest version. In practice, this assumption is rarely valid.

Independent deployment is one of the key advantages of microservices, and compatibility is what preserves that independence. Without it, teams become tightly coupled, and deployments require coordination, defeating the purpose of a distributed architecture.

Safe Schema Evolution

Schema evolution is ultimately about maintaining trust between producers and consumers. Producers must continue evolving to meet business needs, while consumers must retain the freedom to upgrade on their own timelines.

Not all schema changes carry the same level of risk. Some changes are generally safe and can be introduced with minimal impact, while others are inherently breaking and require careful planning. Understanding the difference between these types of changes is essential for preventing production failures.

Adding Optional Fields

Adding optional fields is usually a safe way to evolve a schema. It allows new functionality to be introduced without disrupting existing consumers.

For example:

{
  "orderId": "ORD-1001",
  "customerId": "CUS-501"
}

can evolve into:

{
  "orderId": "ORD-1001",
  "customerId": "CUS-501",
  "currency": "USD"
}

Older consumers will ignore the new field, while newer consumers can take advantage of it. This approach supports gradual adoption and minimizes risk.

Removing Fields

Removing fields is significantly more dangerous. If any consumer depends on a field, its removal will cause failures.

For instance, if a Billing service relies on:

{
  "totalAmount": 249.99
}

and that field is removed, the service will break. Because it is often difficult to know all consumers of an event, it is safest to assume that every published field is in use somewhere.

Renaming Fields

Renaming fields may appear harmless, but it effectively behaves like removing one field and adding another. This makes it a breaking change.

For example:

{
  "customerId": "CUS-501"
}

changing to:

{
  "accountId": "CUS-501"
}

will cause existing consumers to fail because they no longer recognize the expected field. Renaming should therefore be treated with the same caution as removing fields.

Changing Data Types

Changing the data type of a field can also introduce subtle but serious issues. Even if serialization succeeds, consumers may fail when processing the data.

For example:

{
  "quantity": 5
}

becoming:

{
  "quantity": "5"
}

may not immediately cause errors during transmission, but it can break downstream logic that expects a numeric value. Type changes should be treated as breaking changes and handled carefully.

Versioning Strategies

There are situations where compatibility alone is not sufficient, and the contract must change in a way that cannot be made backward-compatible. In such cases, versioning becomes necessary.

Version Inside the Event

One approach is to include version information directly within the event metadata:

{
  "eventType": "OrderConfirmed",
  "eventVersion": "2.0",
  "payload": { ... }
}

This allows consumers to adjust their behavior based on the version while keeping the event name consistent. It provides flexibility but requires consumers to handle multiple versions within their logic.

Different Event Types

Another approach is to define separate event types for each version:

• OrderConfirmedV1
• OrderConfirmedV2

In this model, consumers subscribe only to the versions they support. The producer may need to maintain multiple versions during the transition period, but this approach simplifies consumer logic by avoiding conditional handling within a single event type.

Which Strategy Is Better?

There is no universally correct choice between these strategies. Embedding versions keeps naming simpler, while separate event types make changes more explicit. The most important factor is consistency. Teams should adopt a standard approach across services to reduce confusion and operational complexity.

Avoid Breaking Changes Whenever Possible

Breaking changes introduce tight coupling between services by forcing coordinated deployments. Instead of allowing independent evolution, they create dependencies that can slow down development and increase risk.

Consider the following structure:

Producer
    |
    +-----------------------------+
    |       |       |       |    |
    v       v       v       v    v
Service Service Service Service Service
   A       B       C       D       E

If the producer removes a field, every consumer must update before the change can be safely deployed. Deployment order becomes critical, and the independence of services is lost. Even a single breaking change can undermine the benefits of an event-driven architecture.

Deprecation Is Usually Better Than Removal

When a field becomes obsolete, it is better to deprecate it rather than remove it immediately.

{
  "legacyField": "..."
}

Keeping the field while marking it as deprecated allows consumers time to migrate at their own pace. Once all consumers have transitioned away from the field, it can be safely removed. This gradual approach reduces disruption and maintains system stability.

Common Versioning Mistakes

Certain mistakes appear frequently when teams manage schema evolution.

Treating Events Like Internal DTOs

Internal data transfer objects (DTOs) often change rapidly as implementation details evolve. Public event contracts, however, should be treated with much greater care. They represent agreements between services and should not be modified casually.

Releasing Breaking Changes Without Visibility

Producers often lack visibility into who consumes their events. Making breaking changes without understanding downstream dependencies introduces significant risk. Contract testing can help address this issue by providing insight into how events are used.

Versioning Every Small Change

Some teams create new versions for every minor change:

• OrderConfirmedV2
• OrderConfirmedV3
• OrderConfirmedV4

In many cases, this is unnecessary. Adding optional fields often preserves compatibility without requiring a new version. Excessive versioning can make systems harder to maintain and understand.

Forgetting That Old Events Continue to Exist

Events are often stored for long periods and may be replayed for analytics, auditing, or recovery purposes. Schema evolution must account for both new and historical events. Even if the schema changes, historical data remains unchanged. Systems must be able to handle both.

In the next section, we will explore contract testing and how it helps validate these assumptions before changes reach production.