API Concurrency with ETag-Based Optimistic Locking Strategies

Last month, we pushed a fix for a "lost update" bug that had been plaguing our checkout service for weeks. Two users were hitting the same resource simultaneously, and the last write was consistently clobbering the first, leading to a nightmare of reconciliation scripts. We needed a way to ensure that updates only happen if the data hasn't changed since the client last fetched it, and that’s where ETag-based optimistic locking saved the day.

Why API Concurrency Matters

In a distributed system, you can’t rely on local memory or simple database transactions to handle state across multiple requests. When you build a REST API, you're often dealing with stateless clients that fetch a resource, modify it, and send it back. If another process modifies that resource in the interim, the original client’s update is essentially a blind overwrite.

We first tried using timestamps to track versions, but that broke the moment we had clock skew issues across our nodes. We then experimented with version fields in the JSON body, which worked, but it forced us to change our schema for every single entity. That’s when we moved to standard HTTP headers.

Implementing Optimistic Locking with ETags

The beauty of using ETag (Entity Tag) for API concurrency is that it leverages the HTTP spec rather than reinventing the wheel. An ETag is essentially a hash or a version identifier for a specific state of a resource.

Here is the workflow we implemented:

1. GET Request: The server returns the resource with an ETag header.
2. Client Stores: The client keeps this ETag.
3. PUT/PATCH Request: The client sends the update back with an If-Match header containing that ETag.
4. Validation: The server calculates the current ETag for the resource. If it doesn't match the If-Match header, the server returns a 412 Precondition Failed.

The Code Implementation

In our Go-based microservice, the logic looks roughly like this:

Go
func UpdateResource(w http.ResponseWriter, r *http.Request) {
    resourceID := r.URL.Query().Get("id")
    currentETag := fetchETagFromDB(resourceID) // e.g., "v123"
    
    ifMatch := r.Header.Get("If-Match")
    
    // Check if the client's version is outdated
    if ifMatch != currentETag {
        w.WriteHeader(http.StatusPreconditionFailed)
        return
    }

    // Proceed with the update...
}

This approach is lightweight and avoids the overhead of distributed locking in WordPress or other heavy locking mechanisms. By keeping the logic at the request level, we keep our database connections brief and prevent long-held locks that kill throughput.

Handling Trade-offs and Edge Cases

Is Optimistic Locking a silver bullet? Not exactly. The biggest trade-off is the burden it places on the client. If a client receives a 412, they are forced to re-fetch the resource, merge their changes, and try again. If your UI logic isn't set up to handle these retries, you’ll end up with frustrated users.

We also found that if you are doing heavy state mutations, it’s often safer to implement dry-run modes before committing the actual write. This allows the client to validate their changes against the current state before they even attempt the locked update.

If you’re worried about complex state transitions, you might also want to look into API architecture audit logs to track how the resource arrived at its current ETag. This makes debugging those inevitable "why did my update fail" support tickets much easier.

FAQ: Common Pitfalls

Q: Should I use ETags for all requests? A: No. Use them for state-changing operations (PUT, PATCH, DELETE) where data consistency is critical. Adding them to every GET request can add unnecessary headers and logic complexity.

Q: What if I have a high-traffic resource? A: If the resource changes every few milliseconds, clients will constantly get 412 errors. In those cases, you might need to rethink your API design—perhaps by moving to a more granular resource structure or using a different concurrency pattern like last-write-wins if the business logic allows it.

Q: Does ETag replace versioning? A: Not at all. We still use custom request headers for versioning to manage our API evolution. Think of ETags as resource-level state control and headers as service-level contract control.

Final Thoughts

We’ve seen about a 30% reduction in data corruption bugs since shifting to this model. It’s not perfect; it requires discipline from the frontend team and thoughtful error handling. However, compared to the alternative of "last-write-wins" or complex distributed mutexes, it’s a standard, battle-tested way to handle concurrency.

Next time, I’d probably look into implementing a Weak ETag for resources that don't need byte-for-byte identity, but for now, the strict If-Match check is keeping our state consistent. Just remember: if you don’t manage your concurrency, the distributed nature of your system will eventually manage it for you—usually in the middle of the night.