WordPress Performance: REST API Caching via Stale-While-Revalidate

Last month, I spent about three days debugging a REST API endpoint that was choking under moderate traffic. The server was hitting a wall because every request triggered a heavy WP_Query call, causing spikes in database CPU usage and a TTFB (Time to First Byte) that climbed well over 800ms during peak hours.

If you’re building plugins that serve dynamic data via the REST API, you’ve likely felt this pain. Standard object caching helps, but when you need to serve data at scale, you need a smarter approach. Implementing Stale-While-Revalidate (SWR) allows your plugin to serve a cached response immediately while refreshing that data in the background.

The Problem with Traditional Caching

My first instinct was to use standard wp_cache_set with a short TTL. It worked, but it created a "thundering herd" problem. When the cache expired, the first incoming request would hang while WordPress rebuilt the response, causing a noticeable delay for that user.

I then tried using a transient with a background cron task. That was a disaster; the cron was unreliable, and users often saw stale data for way too long. I needed a pattern where the "revalidation" happens as a direct consequence of a request, but without blocking the user.

Implementing Stale-While-Revalidate for REST API

To implement SWR in WordPress, we need to decouple the response delivery from the data fetching process. We can use a custom cache layer that checks if the data is "stale" (past its TTL) but still usable. If it's stale, we serve it anyway and trigger an asynchronous process to fetch fresh data.

Here’s a simplified architectural pattern for your plugin:

PHP
function get_cached_api_response($key, $callback, $ttl = 3600) {
    $cached_data = get_transient($key);

    if (false !== $cached_data) {
        #6A9955">// Check if we need to revalidate
        if (is_stale($key)) {
            #6A9955">// Trigger background update
            wp_remote_post(home_url('/wp-json/my-plugin/v1/revalidate'), [
                'blocking' => false,
                'body' => ['key' => $key]
            ]);
        }
        return $cached_data;
    }

    #6A9955">// Cache miss: must fetch synchronously
    $data = $callback();
    set_transient($key, $data, $ttl + 86400); #6A9955">// Keep longer for SWR
    return $data;
}

This approach significantly improves WordPress performance by ensuring that the user almost never waits for a cold database query.

Scaling with High-Concurrency Architecture

When your traffic grows, even the background request can become a bottleneck. I’ve found that combining SWR with database-level request coalescing for REST API creates a much more resilient system. By coalescing, you ensure that if ten users hit a stale endpoint simultaneously, only one background fetch actually executes.

For truly high-concurrency architecture, you should consider these layers:

1. L1 Cache (Object Cache): Use Redis or Memcached to store serialized JSON.
2. L2 Cache (SWR Logic): The logic inside your controller that decides whether to return stale data or trigger a fetch.
3. L3 Cache (Edge/CDN): If your API is public, leverage Cache-Control: stale-while-revalidate headers to offload the work to your CDN.

Handling Mutations and Consistency

A common concern with SWR is data consistency. If you update a post, you need to invalidate the cache. I usually hook into save_post or updated_post_meta to clear the specific transients. If you're doing complex mutations, ensure you've read up on WordPress REST API idempotency so that your cache invalidation logic doesn't trigger race conditions.

FAQ: Common Implementation Hurdles

Q: Does SWR increase database load? A: It can, because you're fetching data more often than a strict TTL. However, it shifts that load to background processes, protecting the user experience.

Q: How do I handle authentication with SWR? A: Be careful. Never cache private, user-specific data using this pattern. SWR is best for public or shared data.

Q: What if the background fetch fails? A: The user is still served the stale data, which is better than a 500 error. For critical data, use a circuit breaker to stop attempts if the external service or database is down.

Moving Forward

I'm still experimenting with using Action Scheduler for the revalidation task instead of wp_remote_post. While wp_remote_post is easier to implement, it can sometimes be blocked by server-side firewall rules or loopback limitations.

Regardless of the tool, the goal remains the same: keep the REST API fast by never making the user wait for the database. If you’re looking to push your site further, remember that Service Workers can handle this pattern on the client side, effectively giving you a two-pronged caching strategy. It’s messy, it requires careful cache invalidation, but the performance gains are worth the complexity.