Laravel storage optimization: Implementing content-addressable storage

Last month, I spent three days digging through a client’s S3 bucket that had ballooned to 4TB. We were paying for thousands of identical images uploaded by users across different departments. It was a classic "write-once, read-many" scenario gone wrong, and it was killing our storage budget.

We needed to shift from path-based storage to Laravel storage using a content-addressable approach. Instead of saving a file based on a user-provided name, we save it based on its cryptographic hash. If the file already exists, we simply point to the original. This is the core of content-addressable storage, and it’s a game-changer for infrastructure costs.

Why Content-Addressable Storage Matters

In a standard Laravel setup, Storage::put('avatars/user-1.jpg', $content) creates a new file every time. If a user uploads the same profile picture five times, you store it five times.

When you implement content-addressable storage, you calculate the SHA-256 hash of the file content first. You then store the file at a path derived from that hash, like files/a1/b2/c3/a1b2c3d4...jpg. If the file is already there, you don't upload it again. You just return the existing path.

This isn't just about disk space. It’s a massive win for CDN integration. Since the filename is tied to the content, you can set Cache-Control: immutable headers indefinitely. If the file changes, the hash changes, the path changes, and you get automatic cache busting. It’s cleaner than appending ?v=1 to query strings.

The Implementation Strategy

We first tried building a custom wrapper around the League\Flysystem interface. It was a mistake. We ended up fighting the internal path normalization, which broke our S3 ACLs. We eventually pivoted to a dedicated "Storage Service" pattern that acts as a gatekeeper for all file writes.

Here is how we handle it now:

PHP
namespace App\Services;

use Illuminate\Support\Facades\Storage;
use Illuminate\Http\UploadedFile;

class ContentAddressableStore
{
    public function store(UploadedFile $file): string
    {
        $hash = hash_file('sha256', $file->getRealPath());
        $path = $this->getPathFromHash($hash);

        if (!Storage::exists($path)) {
            Storage::put($path, file_get_contents($file->getRealPath()));
        }

        return $path;
    }

    private function getPathFromHash(string $hash): string
    {
        #6A9955">// Split hash into folders to avoid directory limits
        return 'storage/' . substr($hash, 0, 2) . '/' . substr($hash, 2, 2) . '/' . $hash;
    }
}

This approach ensures that every file is unique on disk. If you’re struggling with database bottlenecks while managing these metadata records, ensure you're eliminating N+1 queries in Eloquent when fetching file metadata for your index views.

Handling CDN Integration

Once your files are stored by hash, your CDN integration becomes trivial. Because the file path is unique to the content, you can safely set long-term cache headers.

In Laravel, you can configure your filesystems.php to handle the URL generation. If you're using CloudFront or Cloudflare, point the origin to your S3 bucket and use the path generated by the ContentAddressableStore service.

One caveat: you need to handle file extensions. Hashes don't contain them. We store a mapping in our database:

When serving the file, we append the extension back to the URL (e.g., cdn.example.com/a1/b2/a1b2...jpg). This keeps the browsers happy and ensures correct MIME type detection.

Performance and Pitfalls

Is this approach overkill for a small blog? Probably. But if you’re scaling, the file system optimization is worth the complexity.

We saw a roughly 30% reduction in storage costs within the first month. However, realize that you lose the ability to easily "delete" a file. Since multiple database records might point to the same hash, you need a reference counting system or a garbage collection job that runs periodically to find hashes with zero references in your database.

If you’re looking to further optimize your infrastructure, keep an eye on modern Laravel trends that focus on offloading these heavy tasks to background workers. Don't try to hash a 50MB file during the request cycle; use a queued job to process the upload and update the database record once the file is safely stored.

Frequently Asked Questions

Does this break Laravel's Storage::url()? No, but you'll need to override the logic that generates the URL to ensure it includes the file extension.

What happens if two different files have the same hash? The probability of a SHA-256 collision is effectively zero for standard web applications. You're safer focusing on application-level bugs.

How do I handle file updates? You don't "update" a file in CAS. You store the new version as a new object and update your database reference to point to the new hash.

I’m still not 100% satisfied with our current garbage collection strategy. We’re currently running a daily cleanup job, but it’s a bit heavy on the DB. I’m considering moving to a TTL-based approach where files without a reference for 30 days are automatically purged, but that requires more complex metadata tracking. It's a work in progress.