Linux Kernel Tuning: Fixing Socket Exhaustion in Docker Proxies

I remember the first time a production proxy keeled over during a routine traffic spike. We were running a standard Nginx-in-Docker setup, and suddenly, every request began timing out with 502 Bad Gateway errors. A quick netstat check revealed thousands of sockets stuck in TIME_WAIT status, effectively starving the system of ephemeral ports.

It’s a classic bottleneck. If you're running high-concurrency Docker proxies, you're bound to hit the limits of the default networking stack eventually. I've spent plenty of time debugging these issues, and the fix almost always comes down to smarter Linux kernel tuning.

Why Your Proxy Is Starving for Sockets

When a client closes a TCP connection, the server keeps the socket in TIME_WAIT for a period—usually 60 seconds—to ensure any delayed packets are handled correctly. In a high-traffic environment, you generate these connections faster than the kernel can recycle them. Once you run out of available ephemeral ports, the kernel simply refuses to open new connections.

Before you start tweaking, ensure your underlying networking isn't suffering from other silent issues. I’ve often seen teams chase socket exhaustion while they actually had Docker networking latency: Debugging with eBPF and tcpretrans hiding in the background. It’s always worth verifying your packet loss metrics before diving into sysctl changes.

The First Attempt: A Risky Shortcut

My first reaction years ago was to indiscriminately lower tcp_fin_timeout. While it helped, it caused issues with some clients that had flaky connections, leading to dropped packets and retransmissions. It wasn't the robust fix I needed.

Instead, we need to focus on recycling sockets more efficiently and expanding the pool of available ports. Here is the standard approach for modern Linux systems.

Essential Linux Kernel Tuning for TCP

You’ll want to modify your /etc/sysctl.conf file to apply these changes persistently. These settings allow the kernel to reuse TIME_WAIT sockets for new connections—a massive win for proxy servers.

Bash
# Increase the range of ephemeral ports
net.ipv4.ip_local_port_range = 1024 65535

# Enable fast recycling and reuse of TIME_WAIT sockets
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 0 # Note: Don't use this on modern kernels (3.12+)

# Increase the maximum number of connections in the listen queue
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

A quick note on tcp_tw_recycle: don't use it. It’s been deprecated for years because it breaks connections from clients behind NATs. Stick to tcp_tw_reuse and properly expanding your port range.

Managing Docker Networking Complexity

When you're dealing with Docker, remember that your host's networking stack is shared with your containers. If you haven't already, you should look into Linux Performance: Cgroups v2 and Systemd Slices for VPS to ensure your proxy container isn't being throttled by the host's resource limits.

Also, watch out for entropy exhaustion. If your proxy is doing heavy SSL/TLS termination, the container might be waiting on the host's /dev/random. If you see performance dips during high traffic, check out Linux performance: Managing Entropy Issues in Docker Containers to see if you need to switch to non-blocking random sources.

Monitoring the Impact

After applying these changes, don't just walk away. You need to see if the TIME_WAIT count actually stabilizes. I usually keep a terminal open with this command to watch the state transitions:

Bash
watch -n 1 "ss -ant | awk '{print \$1}' | sort | uniq -c"

If you see the number of TIME_WAIT sockets holding steady rather than climbing indefinitely, you’re in good shape.

Final Thoughts

Is this a silver bullet? Not exactly. If you reach a point where you're consistently hitting these limits despite tuning, you're likely looking at an architectural wall. At that point, no amount of Linux kernel tuning will save you; you’ll need to scale horizontally by adding more proxy nodes behind a load balancer.

I’m still experimenting with eBPF-based load balancers like Cilium to see if we can bypass the standard netfilter stack entirely. It’s significantly more complex to manage, but for high-concurrency environments, it might be the next logical step. For now, keep your sysctl settings clean, monitor your socket states, and don't be afraid to pull the trigger on scaling out if the host starts sweating.