Data Fetching Architecture: Reducing Latency with Batching and Streaming
Master data fetching performance architecture by implementing request batching and concurrent streaming. Learn how to eliminate bottlenecks and speed up your app.
When I started auditing our dashboard’s performance, I realized we were treating the network like a drive-thru window: one order at a time, waiting for the previous request to finish before even thinking about the next. The "waterfall" effect was killing our INP (Interaction to Next Paint) scores, leaving users staring at loading spinners for upwards of 800ms.
We were making four distinct API calls in the browser to populate a single view. By the time the fourth request returned, the user had already tried to click a filter button, only to be met with a frozen main thread. We needed a shift in our data fetching architecture.
Moving Beyond Sequential Requests
The first trap we fell into was assuming that fetching data "as needed" was efficient. We had components triggering their own useEffect hooks, which meant the browser didn't know about the full data requirement until the initial render finished. This is exactly why mastering Next.js App Router data fetching: avoiding performance waterfalls is so critical; if you aren't parallelizing, you’re losing.
We first tried moving all fetches to the top-level layout, but that just created a massive, blocking payload. We had to rethink how we moved data from the server to the client.
Implementing Request Batching for Efficiency
If you’re still firing individual HTTP requests for small pieces of metadata, you’re wasting overhead on TCP handshakes and header parsing. We implemented a simple API request batching: reduce network overhead and latency pattern using a single GraphQL endpoint to consolidate our calls.
By grouping three small configuration requests into one, we dropped our total round-trip time (RTT) from roughly 450ms to about 120ms. The key wasn't just the batching; it was the ability to define a clear dependency graph before the browser even parsed the HTML.
The Performance Architecture Comparison
| Strategy | Latency Impact | Complexity | Best For |
|---|---|---|---|
| Sequential Fetching | High (Waterfall) | Low | Simple, static pages |
| Request Batching | Low (Reduced RTT) | Medium | Metadata and small lookups |
| Concurrent Streaming | Very Low (TTFB) | High | Large, heavy data sets |
Concurrent Streaming: The Real Game Changer
Batching helps, but it doesn't solve the "all or nothing" problem. If one part of your data is slow, the whole UI waits. We shifted to concurrent streaming, allowing the server to push parts of the document as they become ready.
This is where Next.js streaming SSR: architecting progressive payload serialization becomes essential. Instead of waiting for a 2-second database query, we stream the shell of the page immediately and inject the data asynchronously.
Flow diagram: Client Request → Server Shell Rendering; Server Shell Rendering → Stream Initial HTML; Stream Initial HTML → Async Data Fetching; Async Data Fetching → Stream Data Chunks; Stream Data Chunks → Client Hydration
By prioritizing the "Critical Path"—the elements the user sees first—we kept the interface interactive while the heavier data chunks trickled in. We saw our LCP (Largest Contentful Paint) drop by about 300ms, and more importantly, the browser wasn't choked by a single, massive JSON blob.
Trade-offs and Lessons Learned
It wasn't all smooth sailing. We initially tried to stream everything, which led to excessive "layout shift" as components popped into view. We had to carefully balance Suspense boundaries to ensure we were streaming meaningful chunks rather than just random fragments.
If I were starting this project today, I’d focus on these three rules:
- Never chain: If two fetches aren't dependent, they should be fired concurrently.
- Batch the small stuff: Don't waste network resources on individual requests for configuration or user preferences.
- Stream the big stuff: Use streaming to push the UI shell first, then fill in the heavy content.
We’re still tweaking our cache headers to ensure that batched responses don't invalidate too aggressively, but the current setup feels significantly more robust. Performance architecture is rarely about finding a "silver bullet"—it's about removing the friction you've inadvertently built into your own pipelines.
FAQ
Does request batching make debugging harder? It can. You lose the ability to see individual request failures in the browser’s Network tab easily. We solved this by implementing structured logging on the server to map batched IDs back to individual errors.
Is concurrent streaming overkill for small apps? Probably. If your data fetching is fast and your payload is tiny, the added complexity of managing stream state in your application might not be worth the marginal gains.
How do I know if I have a waterfall problem? Open your Chrome DevTools, go to the Network tab, and look for a "staircase" pattern in the waterfall view. If one request starts only after another finishes, you’ve got work to do.