Docker build cache: How to Audit and Clean for Faster CI/CD

When our CI/CD pipeline started taking over 12 minutes to build a standard microservice, I knew we had a problem with our container infrastructure. We were essentially rebuilding the entire world on every commit, ignoring the fact that most of our dependencies hadn't changed in weeks.

If you’re tired of watching your build logs scroll through redundant package installations, it’s time to take a hard look at your docker build cache. Properly managing these layers is the difference between a 30-second deployment and a coffee-break-sized wait.

Why Your Docker Build Cache Fails

The most common mistake I see is ordering Dockerfile instructions in a way that invalidates the cache unnecessarily. Docker caches layers based on the content of the commands and the files they copy. If you put COPY . . before your RUN npm install or pip install, every single file change—even a typo fix in a README—invalidates the cache for your dependencies.

We once spent about two days debugging why our production builds were consistently slow, only to realize we were invalidating our node_modules layer every time we updated a CSS file. We fixed this by splitting the COPY commands:

Dockerfile
# Copy only package files first
COPY package.json package-lock.json ./
RUN npm ci

# Copy the rest of the application
COPY . .

This simple change allowed Docker to reuse the layer containing our heavy dependencies, shaving roughly 1.8x off our total build time.

Auditing for Docker Image Bloat

Before you start deleting things, you need to see what’s actually taking up space. Docker image bloat often hides in layers that aren't even being used by the final image.

I use docker system df to get a high-level view of my disk usage. It’s a great starting point to see how much space is tied up in "reclaimable" cache.

If your build cache is ballooning, it’s likely because you have dangling layers from failed builds. You can inspect specific layers using docker history <image_name>, which helps you identify which RUN command is adding the most megabytes to your final artifact. Much like we discussed in Docker Image Optimization: Reduce Build Size and Speed Up Deployments, multi-stage builds are your best friend here.

How to Effectively Use Docker Prune

When the disk is full and CI/CD starts throwing no space left on device errors, it's tempting to run docker system prune -a and walk away. Don't do that on a shared build agent—you'll wipe out the cache for every other project on that machine.

Instead, be surgical with your cleanup:

1. Prune dangling build cache: docker builder prune
2. Remove specific old images: docker rmi $(docker images -f "dangling=true" -q)
3. Targeted volume cleanup: docker volume prune

If you are using BuildKit (which you should be, as it’s the default in modern versions), you can leverage the —filter flag to keep cache from the last 24 hours while clearing out the older, stale layers.

Moving Beyond Simple Cleanup

If you've already optimized your layers and cleared the cache but performance is still lagging, consider how your dependencies are structured. Just as I mentioned in Bundle size optimization: Auditing Dependency Graphs for Faster Loads, bloated dependency trees slow down both your application runtime and your container build time.

Here is a quick mental model of the build flow:

Flow diagram: Source Code → Dependency Change?; B -- Yes → Rebuild Deps Layer; B -- No → Use Cached Layer; Rebuild Deps Layer → Build Application; Use Cached Layer → Build Application; Build Application → Final Image

FAQ

Q: Does docker builder prune delete my production images? A: No. It only removes dangling or unused build cache layers. Your tagged images remain safe.

Q: How often should I run a prune command in CI/CD? A: I recommend running a targeted docker builder prune --filter "until=24h" as a cron job or a cleanup step in your pipeline once a day to keep the build agent's disk usage stable.

Q: Is there a way to force a full rebuild without clearing the cache? A: Yes, use the --no-cache flag during your docker build command. Use it sparingly, as it completely bypasses the benefits of layer caching.

Final Thoughts

Managing the docker build cache isn't a "set it and forget it" task. It requires a disciplined approach to Dockerfile design—specifically keeping your frequent changes at the bottom of the file. Next time, I’m planning to experiment with remote cache backends (like GitHub Actions cache or Registry cache) to see if we can share layers across different build agents. For now, keep your layers small, your COPY commands specific, and don't be afraid to run a targeted prune.