Liveness Probes: Automating Container Reliability in Kubernetes

Previously in this course, we explored troubleshooting Pod crashes, where we learned how Kubernetes identifies when a process exits with an error. But what happens when your application doesn't exit, but simply stops responding?

In this lesson, we address this "zombie" state. You’ll learn how to implement Liveness Probes to ensure your application remains healthy by forcing Kubernetes to restart containers that have entered a deadlocked or unresponsive state.

Why Liveness Probes Are Essential for Reliability

By default, Kubernetes considers a Pod "healthy" as long as the process inside the container is running. If your web server enters a deadlock—perhaps due to a thread pool exhaustion or a circular dependency—the process remains active, but it can no longer serve traffic.

Kubernetes has no way of knowing your app is stuck unless you explicitly tell it how to check. A Liveness Probe acts as a heartbeat monitor. If the probe fails repeatedly, the kubelet kills the container and restarts it according to your Pod's restart policy, effectively providing a self-healing mechanism for your infrastructure.

How to Configure a Liveness Probe

To add a Liveness Probe, you define it directly within the containers section of your Pod manifest. The most common method is an httpGet probe, which instructs Kubernetes to periodically request a specific endpoint on your application.

Here is a manifest for a simple web application with a configured liveness probe:

YAML
apiVersion: v1
kind: Pod
metadata:
  name: liveness-demo
spec:
  containers:
  - name: nginx-app
    image: nginx
    livenessProbe:
      httpGet:
        path: /healthz
        port: 80
      initialDelaySeconds: 3
      periodSeconds: 5

Breaking Down the Configuration

• httpGet: Tells Kubernetes to perform an HTTP GET request.
• path: The endpoint the probe will hit. Your application must have a route (like /healthz) that returns a 200–399 status code.
• initialDelaySeconds: How long the kubelet waits after the container starts before performing the first probe. This prevents the probe from killing a container that is still booting up.
• periodSeconds: How often the probe runs.

While HTTP probes are standard for web apps, you can also use exec probes to run commands (like checking a file existence) or tcpSocket probes to verify a specific port is accepting connections.

Hands-on Exercise: Triggering a Restart

We will now modify your running project to include a liveness check.

1. Create the manifest: Create a file named liveness-pod.yaml using the configuration above.
2. Apply the manifest: Run kubectl apply -f liveness-pod.yaml.
3. Inspect the probe: After the pod is running, run kubectl describe pod liveness-demo. Look under the "Events" section; you will see the probe status.
4. Force a failure: In a real scenario, you would delete your /healthz route or make it return a 500 error. Watch the Pod status: Kubernetes will eventually mark the probe as failed and restart the container.

Common Pitfalls to Avoid

• Setting the initial delay too short: If your app takes 10 seconds to start but your initialDelaySeconds is 3, Kubernetes will kill the container before it’s even ready, creating an infinite crash loop.
• Heavy health checks: Don't make your /healthz endpoint perform complex database queries or external API calls. If the dependency is slow, your probe will fail, causing unnecessary restarts. Keep it lightweight—see Express Middleware Health Check for tips on designing efficient health endpoints.
• Ignoring the difference between Liveness and Readiness: A liveness probe is for restarts. We will cover Readiness—which controls traffic flow—in the next lesson.

FAQ: Frequently Asked Questions

Does a Liveness Probe replace crash detection? No. Kubernetes still detects if a process exits (returns a non-zero code). The probe is strictly for processes that are running but "stuck."

What happens if the probe fails? The kubelet kills the container and starts a new one based on the restartPolicy (which defaults to Always).

Can I use a shell script for the probe? Yes, use the exec block in the manifest:

YAML
livenessProbe:
  exec:
    command:
    - cat
    - /tmp/healthy

Recap

We’ve moved beyond simple process monitoring to self-healing infrastructure. By adding a livenessProbe to your Pod manifests, you ensure that even if your application code deadlocks, Kubernetes will intervene to restore service. This is a foundational step in building reliable distributed systems.

Up next: Readiness Probes — we’ll learn how to prevent traffic from hitting containers that aren't ready to serve requests yet.