Rate limiting with distributed locks: Stopping Redis-backed DoS

During a recent refactor of a high-traffic gateway service, my team realized that our rate limiting strategy was actually making us more vulnerable to a denial of service. We had implemented a distributed locking mechanism to ensure atomic increments in Redis, but we didn't account for the latency spikes that happen when hundreds of processes fight for the same key.

If your API relies on Redis for traffic control, you're likely using some form of distributed locking to manage state. When done incorrectly, this setup becomes a bottleneck that crashes under load.

The Problem with Naive Distributed Locking

When we first built our rate limiter, we used a standard SETNX (set if not exists) pattern in Redis to lock a user's ID while updating their request count. It looked something like this in our Node.js middleware:

JAVASCRIPT
// A naive approach that leads to lock contention
async function rateLimit(userId) {
  const lockKey = CE9178">`lock:rate:${userId}`;
  const acquired = await redis.set(lockKey, CE9178">'locked', CE9178">'NX', CE9178">'PX', 100);
  
  if (!acquired) {
    throw new Error(CE9178">'Too many requests');
  }
  // ... proceed to increment count
}

The logic felt sound during local testing. However, in production, we saw the lockKey contention spike during traffic bursts. Because we were using a hard 100ms lock, any process that couldn't acquire the lock immediately would fail, effectively turning a "slow down" instruction into a "hard block." We were causing our own denial of service because the lock itself became the primary point of failure under stress.

Understanding Distributed Locking Risks

When you implement rate limiting with distributed locking, you're essentially creating a synchronous dependency on a network-attached store. If your Redis instance experiences even a minor network hiccup or garbage collection pause, every request hitting that lock becomes a queued process.

This is a classic denial of service vector. If an attacker knows your locking strategy, they can trigger artificial contention by sending high-frequency requests from multiple IPs that resolve to the same internal resource key.

Better Patterns for Redis Security

Instead of locking the entire operation, we shifted to atomic Lua scripts. By pushing the logic into Redis, we eliminate the need for a client-side lock entirely. This is a core tenant of api security—keep the critical path as short as possible.

Here is how we refactored the logic to avoid distributed locking contention:

LUA
-- rate_limit.lua
local current = redis.call("INCR", KEYS[1])
if current == 1 then
    redis.call("PEXPIRE", KEYS[1], ARGV[1])
end
return current

By calling this script from Node.js (or PHP using eval), the increment and the expiration happen in a single, atomic step on the Redis server. No locks are required, and the CPU overhead on the Redis side is negligible, usually around 0.2ms per execution.

Lessons from the Field

We learned that if you find yourself needing a lock for a simple counter, you're likely over-engineering the persistence layer. Before implementing complex locking, consider these three questions:

1. Does the counter need to be 100% accurate, or is "eventually consistent" acceptable?
2. Can the operation be performed atomically using native Redis commands like INCRBY or HSET?
3. What is the fallback behavior when the lock fails? If it's a 500 error, you've created a vulnerability.

We previously discussed the nuances of Resource Locking: How to Prevent Deadlocks and DoS in Node.js & PHP, where we explored managing shared resources. Rate limiting is just a specialized case of this. If you are working in a multi-tenant environment, make sure to check out Postgres Rate Limiting and Redis Patterns for Multi-Tenant APIs to see how to isolate these counters so that one noisy tenant doesn't crash the lock manager for everyone else.

FAQ

Q: Should I ever use distributed locks for rate limiting? A: Only if you are performing complex, multi-step state changes that cannot be expressed in a single Lua script. For simple request counting, avoid them.

Q: How do I handle Redis downtime in this architecture? A: Always implement a "fail-open" strategy. If your Redis cluster is unreachable, your API should default to allowing the request (or using a local in-memory cache) rather than rejecting all traffic.

Q: Is Lua scripting safe? A: Yes, but keep scripts small. Long-running Lua scripts block the entire Redis server, which creates its own DoS risk.

I'm still not entirely satisfied with our current failover mechanism—if Redis goes down, we lose all rate-limiting state, which allows for a sudden surge of traffic. I'm currently investigating a local LRU cache pattern to act as a buffer, but that introduces its own set of synchronization headaches. For now, the move to Lua scripts has significantly stabilized our throughput.