Nginx performance tuning: Mastering Worker Processes and Buffers

We hit a wall during a flash sale last quarter where our primary load balancer started dropping connections at around 15,000 concurrent users. It wasn't the application code; it was Nginx choking on its default configuration, failing to keep up with the burst of incoming sockets.

If you’re running Nginx in a high-traffic environment, the default settings are almost certainly holding you back. You aren't just tuning a web server; you’re tuning the gateway to your entire infrastructure.

Understanding nginx worker_processes and connections

The most common mistake I see is misconfiguring the concurrency model. Nginx uses an event-driven, asynchronous architecture, but it still relies on OS-level resources.

The worker_processes directive should generally match the number of available CPU cores. You can check this by running grep processor /proc/cpuinfo | wc -l. Don’t set this to "auto" if you are running in a constrained container environment, as it might miscalculate based on the host's total cores rather than your cgroup limit.

NGINX
# nginx.conf
worker_processes auto; # Or set to a specific number of cores
worker_connections 2048; # Needs to be tuned based on your ulimit

However, worker_connections is where most people get tripped up. This number defines how many simultaneous connections a single worker process can handle. The total max connections is worker_processes * worker_connections. If you have 4 cores and 1024 connections, you’re capped at 4096 concurrent connections—hardly "high traffic."

Before increasing this, check your system limit: ulimit -n

If your system limit is lower than your desired worker_connections, Nginx will throw errors in your logs. Update the worker_rlimit_nofile in your main context to match your needs.

Nginx buffer size optimization for throughput

Once your connection handling is solid, you need to look at how data is buffered. If your nginx buffer size optimization is off, Nginx will write request bodies and proxy responses to temporary files on disk, which is a massive performance killer.

You want to keep these operations in RAM. Here is how I typically configure my buffers for a standard API gateway:

NGINX
http {
    client_body_buffer_size 16k;
    client_header_buffer_size 1k;
    client_max_body_size 8m;
    large_client_header_buffers 4 8k;

    # Proxy buffer settings
    proxy_buffer_size 8k;
    proxy_buffers 8 16k;
    proxy_busy_buffers_size 32k;
}

If you see a client request body is buffered to a temporary file in your error logs, your client_body_buffer_size is too small. Bump it up until those messages disappear. Just remember that every connection consumes these buffers, so don't set them to massive values unless you have plenty of RAM.

The high-traffic configuration checklist

When dealing with massive concurrency, it’s not just about the buffers. You need to ensure the OS can handle the sheer volume of sockets.

If you are struggling with database-level bottlenecks while optimizing your web tier, ensure you've also looked at Postgres Partial Indexes: Optimizing High-Traffic Query Performance to keep your backend from being the slowest link in the chain.

What I’d do differently next time

We spent about three days debugging an issue where Nginx was reporting "too many open files." We kept increasing worker_connections without updating the underlying system ulimit. It’s a classic trap. If I were setting this up from scratch today, I’d start by monitoring the active and waiting connections via the stub_status module before making any changes.

Don't optimize for the sake of it. Measure your current buffer usage, check your CPU wait times, and make incremental changes. If you’re handling heavy write loads, consider also reading about Database performance: Implementing Write-Buffer Coalescing for High-Frequency Updates to ensure your database isn't bottlenecking the traffic you've just successfully routed to it.

Frequently Asked Questions

Q: How do I know if my buffer sizes are too small? Check your Nginx error logs. If you see "upstream sent too big header" or "client request body is buffered to a temporary file," your buffers are likely undersized.

Q: Does increasing worker_processes beyond the number of CPU cores help? Generally, no. It creates unnecessary context switching, which hurts performance more than it helps. Stick to the number of physical cores.

Q: What is the most important setting for Nginx high traffic configuration? It’s a balance, but worker_connections combined with ulimit -n is usually the first wall you'll hit. If Nginx can't open enough file descriptors, nothing else matters.