API evolution is safer with the expand-and-contract pattern. Learn how to manage schema migrations and maintain backward compatibility in distributed systems.
Last month, we had to rename a core user identifier field across four microservices. The naive approach—updating the schema and pushing all services at once—would have triggered a cascade of 500 errors. Instead, we relied on the expand-and-contract pattern to migrate the data safely over three days.
In a distributed system, you rarely control all your consumers. When you change an API contract, you aren't just updating code; you're coordinating a state transition across multiple independent deployments. If you break the contract, you break the system.
Most engineers try to solve this with strict versioning, but API Versioning Strategies: Maintaining Backward Compatibility at Scale teaches us that versioning is often a heavy-handed solution. Instead of forcing clients to move to v2, we prefer evolving the schema in place while maintaining compatibility.
The expand-and-contract pattern (often called "Parallel Change") breaks a breaking change into three distinct phases. It forces you to think about your API contract design as a series of additive steps rather than a single atomic swap.
You introduce the new structure while keeping the old one alive. If you are renaming a field from user_id to account_uuid, your service should accept and return both.
JSON// Old schema { "user_id": "123" } // Expanded schema { "user_id": "123", "account_uuid": "123" }
During this phase, your database migration should be additive. Add the new column or field, but don't drop the old one. If you're using a relational database, you might use a trigger or application-level logic to keep the two fields synchronized.
Once the service is "expanded," you update all your consumers. This is the longest phase. You monitor logs and metrics to ensure that clients have moved from the old field to the new one.
We typically set a deadline for this, often around two weeks. If a client is still hitting the old field, you’ll see it in your observability dashboard. It’s a great time to implement API Design Schema Evolution: Managing Changes with Field Projection to verify that your fields are still reachable.
Only after the logs show zero traffic to the legacy field do you "contract." You remove the old field from the API response and perform a final database cleanup. If you haven't verified this, you're just guessing—and guessing in production usually leads to an on-call nightmare.
The biggest mistake I see is rushing the "Contract" phase. Engineers often see the traffic hit zero for a few hours and immediately delete the field. Don't do this.
Instead, keep the legacy field in the code for at least one full deployment cycle. I once had a batch job that ran on a monthly schedule; it didn't show up in our daily traffic logs, and deleting the old field prematurely broke the entire reporting pipeline.
Also, consider your state mutations. When you perform these migrations, ensure your API Design: Implementing Dry-Run Modes for Safe State Mutations are active. This allows you to test the new schema against real production data without actually committing changes to the database.
Sometimes, the overhead of maintaining two schemas is simply too high. If the change is massive—say, changing the entire authentication header format—the expand-and-contract pattern might complicate your codebase more than it helps.
In those rare cases, it's better to stand up a new endpoint and deprecate the old one. Just make sure you communicate the timeline clearly.
Managing API evolution is about reducing the blast radius of your changes. By decoupling your deployment from your migration, you turn a high-risk event into a boring, routine task.
Next time I approach a schema change, I’ll probably spend more time on the automated testing of the "Expand" phase. We spent about two days writing custom integration tests to ensure that the dual-write logic held up under load. It felt like overkill at the time, but it saved us from a production rollback.
What’s the most complex migration you’ve handled? Did you use a similar phased approach, or did you lean on a different strategy?
How do I handle database constraints during the expand phase?
You can't add NOT NULL constraints to a new field until all your producers are writing to it. Keep the new field nullable during the "Expand" phase and only apply the constraint during the "Contract" phase.
Does this pattern increase latency? Yes, slightly. Dual-writing or supporting two schema formats adds overhead. If you're operating at massive scale, profile your services before and after the expansion to ensure the performance hit is within your acceptable budget.
What if I need to roll back? That's the beauty of this pattern. If the new schema causes issues, you simply stop the consumer migration. Your system is already running in a compatible state, so you don't have to revert your database or service code immediately.
API Design Schema Evolution is simpler when you use forward-compatible field projection. Learn how to evolve your REST architecture without breaking clients.
Read moreAPI idempotency prevents duplicate side effects in distributed systems. Learn how to use deterministic correlation IDs to ensure state consistency during retries.
