LLM Cost Control: Implementing Per-User Quotas and Rate Limiting

Last month, a single misconfigured loop in one of our background worker services nearly burned through a third of our monthly OpenAI budget in under four hours. We didn't have any guardrails in place, and the dashboard spike was a wake-up call that "monitoring" isn't the same thing as "enforcement."

If you’re shipping LLM features, you need to treat API tokens like actual currency. This guide covers how to move beyond basic monitoring into active LLM cost control using per-user rate limiting and token-based quotas.

The Problem with "Just Watch It"

Most engineers start by logging every request to a database like Postgres or ClickHouse. While that’s great for retrospective analysis, it’s useless for stopping a runaway process. By the time your dashboard alerts you to a cost anomaly, the damage is already done.

We initially tried wrapping our calls in a simple try-catch block that logged usage to a standard SQL table. The latency overhead was negligible—around 12ms per request—but the race conditions in our distributed environment meant that a user could trigger multiple parallel requests that collectively bypassed our logic. We needed something atomic.

Implementing Atomic Token Usage Tracking

To get this right, you need a centralized, high-performance store. Redis is the industry standard here because it handles atomic increments with ease.

We use a two-tiered approach:

1. Rate Limiting: A hard cap on requests per minute (RPM) to prevent abuse or accidental loops.
2. Token Quotas: A monthly or daily budget based on input + output tokens.

If you’re already familiar with standard API throttling, you can adapt those patterns, but remember that LLM cost control requires tracking consumption, not just hit count. For high-performance, multi-tenant environments, I’ve found that using Laravel Redis Lua scripting for deterministic rate limiting is the most reliable way to ensure your logic is atomic and consistent across multiple application instances.

The Redis Strategy

We store two keys per user:

• rate_limit:{user_id}: A simple counter with a 60-second TTL.
• token_budget:{user_id}:{month}: A counter that tracks the total sum of usage.total_tokens returned by the LLM API.

Here is a simplified snippet of how we check this before firing an API request:

JAVASCRIPT
async function canProcessRequest(userId, estimatedTokens) {
  const redis = getRedisClient();
  const currentUsage = await redis.get(CE9178">`token_budget:${userId}:2023-10`);
  
  if (parseInt(currentUsage) + estimatedTokens > MONTHLY_LIMIT) {
    throw new Error("Budget exceeded");
  }

  // Atomic increment
  return await redis.incrby(CE9178">`token_budget:${userId}:2023-10`, estimatedTokens);
}

Integrating with Middleware

Don't put this inside your business logic. It belongs in your request pipeline. By using middleware, you ensure that every incoming request is validated against your LLM cost control policy before the heavy lifting begins.

If you’re working in a Node.js or Laravel environment, this is a perfect place to leverage JWT security: implementing scope-based validation for APIs to ensure the user identity is verified before you even look up their budget in Redis. This prevents unauthenticated users from forcing your system to perform expensive database lookups.

Handling the "In-Flight" Reality

One thing I learned the hard way: you don't know the exact token usage until the stream finishes. If you wait for the full response to update your Redis counter, you're vulnerable to "draining" attacks where a user fires 50 requests before the first one finishes and updates the counter.

What I'm doing now is "pessimistic estimation." We calculate the max tokens allowed in the prompt and add a buffer for the expected response length. We increment the Redis counter before sending the request. If the user hits the limit, the request is rejected immediately. Once the request finishes, we perform a "correction" by calculating the actual tokens used and adjusting the Redis key:

JAVASCRIPT
const actualUsage = response.usage.total_tokens;
const difference = actualUsage - estimatedTokens;
await redis.incrby(CE9178">`token_budget:${userId}:2023-10`, difference);

Why This Matters

If you don't implement proactive LLM cost control, you are essentially giving your users an open-ended line of credit on your corporate credit card.

I’m still not 100% happy with our "pessimistic estimation" logic because it can feel restrictive for users who are just under the limit. We’re currently exploring API throttling: adaptive backoff strategies for resilient systems to see if we can provide a smoother experience when users hit their quota, perhaps by suggesting they upgrade their tier rather than just throwing a hard 403 error.

Frequently Asked Questions

Q: Does Redis latency impact LLM response time? A: Negligible. Redis operations are sub-millisecond. The bottleneck will always be the LLM provider's TTFT (Time To First Token), not your budget check.

Q: How do you handle multi-model usage? A: We normalize costs into "base tokens" based on the model's price per million tokens. Everything is converted to a unified unit before hitting the Redis quota.

Q: What happens if Redis goes down? A: We have a fail-open policy for the budget check. If the cache is unreachable, we log a critical error to our observability platform but allow the request to proceed to ensure availability.

Building these guardrails is essentially an insurance policy for your infrastructure. I’d rather have a user get a "Budget Exceeded" message than wake up to a $5,000 bill because of an infinite loop. It’s worth the extra complexity.