CronJobs for Scheduling: Automating Batch Tasks in Kubernetes

Previously in this course, we explored how to run finite, one-off tasks using the Introduction to Jobs: Running One-Off Tasks in Kubernetes. While a Job is perfect for a task that needs to run to completion once, many real-world scenarios require repetition—like database backups, generating daily reports, or cleaning up temporary files.

In this lesson, we add the "time" dimension to your automation toolkit by learning how to use a CronJob.

Understanding the CronJob Concept

A CronJob in Kubernetes is essentially a controller that manages a series of Jobs based on a time-based schedule. It follows the standard Linux cron format, which you may have encountered if you've done any Linux cron job automation: scheduling tasks and debugging.

When you define a CronJob, you aren't just saying "run this container"; you are saying "spawn a Job object every X minutes, hours, or days." The Kubernetes controller handles the creation of these Jobs, ensuring that your scheduled task runs exactly when requested.

The Anatomy of a CronJob Manifest

A CronJob manifest contains three critical sections:

1. schedule: The cron syntax (e.g., * * * * * for every minute).
2. jobTemplate: The blueprint for the Pods the CronJob will create.
3. History Limits: Settings to prevent your cluster from being cluttered with thousands of finished Job objects.

Worked Example: A Scheduled Cleanup Task

Let’s create a CronJob that prints "Database cleanup starting..." every minute. This is a classic "hello world" for batch automation.

Create a file named cleanup-cronjob.yaml:

YAML
apiVersion: batch/v1
kind: CronJob
metadata:
  name: database-cleanup
spec:
  # Run every minute
  schedule: "* * * * *"
  # Keep only the last 3 successful jobs to save space
  successfulJobsHistoryLimit: 3
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: cleaner
            image: busybox
            command: ["/bin/sh", "-c", "echo 'Database cleanup starting...'; sleep 5; echo 'Done.'"]
          restartPolicy: OnFailure

Apply this to your cluster:

Bash
kubectl apply -f cleanup-cronjob.yaml

Check the status with kubectl get cronjob. You will see the schedule, the last time it ran, and how many jobs are currently active. After a minute or two, check kubectl get jobs to see the pods the CronJob has spawned.

Managing Job History

In production, you don't want your API server overwhelmed with thousands of completed Job objects. The fields successfulJobsHistoryLimit and failedJobsHistoryLimit in the spec allow you to control how many history records are kept. Always set these to a reasonable number (like 3 or 5) to keep your cluster clean.

Hands-on Exercise

1. Modify your cleanup-cronjob.yaml to run every 5 minutes instead of every minute (Hint: Use */5 * * * *).
2. Update the manifest with kubectl apply.
3. Use kubectl get pods to verify that no new pods are created immediately, confirming the new schedule is in effect.

Common Pitfalls

• Timezone Confusion: Kubernetes CronJobs use the timezone of the controller manager (usually UTC). If your team expects a job to run at 9 AM local time, verify your cluster's timezone settings first.
• Overlapping Jobs: By default, if a job takes longer than the interval, the CronJob will trigger a new one. If your task is heavy, use the concurrencyPolicy: Forbid setting in your spec to prevent multiple instances from running simultaneously.
• The "Job" vs. "CronJob" confusion: Remember that the CronJob is just a factory for Jobs. If you want to debug why a task failed, you need to check the logs of the specific Pods generated by the Job, not the CronJob itself.

FAQ

Q: Can I manually trigger a CronJob? A: Yes! You can trigger a manual run without waiting for the schedule using: kubectl create job --from=cronjob/database-cleanup manual-test-run.

Q: What happens if a job fails? A: The restartPolicy: OnFailure in the Job template ensures that if the container exits with an error, Kubernetes will try to restart the specific container within that Pod.

Q: How do I stop a CronJob? A: You can set suspend: true in the spec and apply the change. This pauses the schedule without deleting the resource.

Recap

We have successfully moved from manual execution to automated scheduling. By wrapping a Job in a CronJob template, we enabled the cluster to handle repetitive maintenance tasks autonomously. We also learned how to manage resource usage by cleaning up old job history.

Up next: We will dive into Persistent Volumes and Claims, where we'll learn how to give our batch tasks and applications a place to store data that survives container restarts.