Implementing Zero-Trust Network Policies with Cilium and Hubble

Why Traditional Network Policies Fail

If you're still relying on IP-based firewall rules in Kubernetes, you're fighting a losing battle. Pods are ephemeral; they spin up, die, and get rescheduled across different nodes constantly. Hardcoding IP ranges in your NetworkPolicy manifests is a maintenance nightmare that breaks the moment a developer scales a deployment.

I’ve spent too many late nights debugging why a microservice couldn't talk to its database because an IP changed. That’s why I moved my production clusters to Cilium. By leveraging eBPF, Cilium operates at the Linux kernel level, allowing us to enforce security based on service identity rather than volatile network addresses.

The Architecture of Zero-Trust

Zero-Trust isn't a product you buy; it's a mindset of "never trust, always verify." In a Kubernetes context, this means every request between pods must be authenticated and authorized.

Cilium achieves this by assigning an identity to each pod based on its labels (e.g., app=billing). When a packet moves, Cilium checks the identity of the source and destination against your policies. If it’s not explicitly allowed, it’s dropped. No exceptions.

Getting Started: Cilium and Hubble Setup

First, ensure you have a cluster running a supported kernel (5.4 or later is recommended for full eBPF features). Install Cilium using Helm 3.12+.

Bash
helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium --version 1.14.5 \
  --namespace kube-system \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true

Once installed, verify the pods are running: kubectl get pods -n kube-system -l k8s-app=cilium

Enforcing Zero-Trust with CiliumNetworkPolicy

Unlike standard Kubernetes NetworkPolicy, Cilium’s CiliumNetworkPolicy supports Layer 7 filtering. You can restrict traffic not just by port, but by HTTP method and path.

Here is a policy that ensures the frontend pod can only GET data from the backend API:

YAML
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "secure-api-access"
spec:
  endpointSelector:
    matchLabels:
      app: backend
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/data"

Apply this with kubectl apply -f policy.yaml. Now, even if the frontend is compromised, an attacker can't POST or DELETE data to your backend.

Observability with Hubble

Implementing policies is only half the battle. You need to verify that your policies are actually working without blocking legitimate traffic. This is where Hubble shines. It gives you deep eBPF observability into your network flow.

To see what’s being dropped in real-time, run: hubble observe --pod backend --verdict DROPPED

This command shows you every connection attempt that hit your policy wall. When I’m rolling out new zero-trust policies, I keep this window open. It turns "why is my app broken?" into "oh, I forgot to allow the health check endpoint."

Hard-Won Lessons

1. Start in Monitor Mode: Don't enforce policies globally on day one. Use Cilium’s policy-audit-mode to log violations without dropping traffic.
2. Identity over IP: Stop thinking about CIDR blocks. Think about labels. If your labels are messy, your security is messy. Clean up your deployment metadata before writing policies.
3. L7 is Power, but Expensive: Layer 7 filtering requires Cilium to inspect the HTTP stream. It’s slightly more resource-intensive than L3/L4 filtering. Use it selectively for your most sensitive services.

Final Thoughts

Transitioning to a zero-trust model using Cilium and Hubble isn't just about security compliance—it's about gaining visibility into what your services are actually doing. By moving security into the kernel, you remove the reliance on iptables and gain a high-performance, identity-aware network that scales with your infrastructure.

Stop trusting your internal network. Start verifying it.