File Integrity Monitoring: eBPF and Fanotify for Docker VPS

Running a production Dockerized VPS means you’re essentially managing a multi-tenant environment where the line between container storage and host files can get blurry. Relying on basic tools like AIDE is a great start—as we covered in Linux Security: File Integrity Monitoring with AIDE and Systemd Timers—but sometimes you need lower-level visibility. When I needed to track exactly which process touched a critical configuration file in real-time, I realized I needed a hybrid approach using eBPF and Fanotify.

Why combine eBPF and Fanotify?

Most monitoring tools rely on inotify, which is excellent for simple watches but fails under high-load scenarios or when you need deep process context. If you want to stop an unauthorized write before it happens, or log the specific PID and UID of the culprit, you need more power.

Fanotify allows us to intercept file access events and even block them until a decision is made. However, Fanotify can be noisy. This is where eBPF comes in; we use eBPF to filter events at the kernel level, only passing the "interesting" ones to our user-space Fanotify daemon. This prevents your monitoring tool from consuming all your CPU cycles.

The architecture of a custom FIM

We aren't building a full-blown security suite here. We’re building a targeted observer.

1. The eBPF Probe: Attaches to sys_enter_openat or similar syscalls. It checks if the file path matches our "watched" list.
2. The User-space Daemon: Uses fanotify_init and fanotify_mark to set up the watch.
3. The Logic: When the kernel triggers an event, the daemon logs the event, container ID (if applicable), and process metadata.

I first tried using pure inotify via a Go wrapper, but it dropped events when our CI/CD pipeline kicked off a massive deployment. Switching to a hybrid eBPF-Fanotify model reduced our event-processing latency by roughly 40ms during peak bursts. It’s a bit more complex to maintain, but it’s the only way to get reliable data without killing the host's performance.

Implementing File Integrity Monitoring

You’ll need bcc or libbpf installed on your host. Here is a skeleton of how we hook the file access:

C
// simplified eBPF skeleton
SEC("tracepoint/syscalls/sys_enter_openat")
int trace_openat(struct trace_event_raw_sys_enter *ctx) {
    char filename[256];
    bpf_probe_read_user(&filename, sizeof(filename), (void *)ctx->args[1]);
    
    // Filter logic: only report if file is in /etc/docker or /var/lib/docker
    if (is_watched_path(filename)) {
        bpf_perf_event_output(ctx, &events, BPF_F_CURRENT_CPU, ...);
    }
    return 0;
}

Once the eBPF probe filters the noise, the user-space component handles the notification. I keep the logic simple: if an unauthorized process touches a file, the daemon sends a message to a local socket. If you’re already using Vector.dev Log Management: Real-Time Routing on Dockerized VPS, you can pipe these logs directly into your existing observability stack.

Addressing the Docker context

One of the biggest hurdles is mapping a container's internal path to the host's overlay filesystem. When a container writes to /etc/nginx/nginx.conf, the host sees it in /var/lib/docker/overlay2/<id>/diff/etc/nginx/nginx.conf.

If you're looking for deeper kernel-level hardening, you might also want to look at Linux kernel security: How to harden your Docker host with LKRG to catch exploits that FIM might miss. My rule of thumb: FIM is for audit trails, while kernel-level hardening is for preventing the initial compromise.

Common pitfalls

• Race Conditions: Fanotify FAN_ALLOW events can cause deadlocks if your user-space daemon hangs. Always use a timeout.
• Overhead: Even with eBPF filtering, don't watch the entire /var/lib/docker tree. You will crash your kernel's event buffer. Be surgical.
• Kernel Versioning: eBPF features evolve quickly. Ensure you’re running at least a 5.x kernel to get the best stability for bpf_probe_read_user.

Frequently Asked Questions

Is eBPF-based FIM faster than standard auditing? Yes, because eBPF runs in the kernel, it filters out millions of irrelevant events before they ever reach your application, whereas standard auditing tools often perform heavy context switching.

Can this replace standard Docker security tools? No. This is a "Defense in Depth" strategy. You should still use seccomp profiles, AppArmor, and read-only filesystems in your containers.

What happens if the eBPF program crashes? The system will continue to run, but you lose monitoring. I recommend wrapping your daemon in a systemd service with a Restart=always policy to ensure it recovers quickly.

I’m still experimenting with how to best correlate these events with specific container orchestration metadata. It’s a work in progress, but the visibility I’ve gained into the host-container boundary has been worth the two weeks of debugging. If you’re just starting, keep your watch list small and iterate.