Mastering Job Chaining and Batching in Laravel

Previously in this course, we covered the basics of Asynchronous Processing with Queues in Laravel. We established how to move time-consuming tasks into the background to keep our API responses snappy. However, real-world applications rarely perform tasks in isolation.

Often, you need a sequence of operations where the output of one job informs the next, or you need to process a large collection of items and perform a final cleanup action only once they all finish. This is where job chaining and batching come into play.

Understanding Job Chaining

Job chaining allows you to specify a sequence of queued jobs that should run in a specific order. If one job in the chain fails, the subsequent jobs are not executed. This is perfect for workflows like "Process Video" -> "Generate Thumbnail" -> "Notify User."

To implement this, you use the Bus::chain() method. Let's look at our project board. Suppose when a user completes a project, we need to generate a summary report, upload it to S3, and then email the project owner.

PHP
use App\Jobs\GenerateProjectReport;
use App\Jobs\UploadReportToS3;
use App\Jobs\NotifyOwnerOfReport;
use Illuminate\Support\Facades\Bus;

Bus::chain([
    new GenerateProjectReport($project),
    new UploadReportToS3($project),
    new NotifyOwnerOfReport($project),
])->dispatch();

If GenerateProjectReport fails, the chain stops. Laravel handles the state transition automatically, ensuring you don't end up with orphaned files or broken notifications.

Orchestrating Workflows with Job Batching

While chains are linear, batching is designed for parallel execution. When you need to process hundreds of tasks—like generating individual PDF exports for every task in a project—dispatching them one by one is inefficient.

Batching allows you to group these jobs and track their overall progress. You can even define callbacks for when the entire batch finishes, fails, or encounters an exception.

Implementing a Job Batch

First, ensure you have a job_batches table in your database. If you haven't run it yet, execute:

Bash
php artisan queue:batches-table
php artisan migrate

Now, let's update our ProjectController to batch task processing:

PHP
use App\Jobs\ProcessTaskExport;
use Illuminate\Support\Facades\Bus;
use Throwable;

$batch = Bus::batch([
    new ProcessTaskExport($task1),
    new ProcessTaskExport($task2),
    #6A9955">// ...
])->then(function ($batch) {
    #6A9955">// All jobs completed successfully
})->catch(function ($batch, Throwable $e) {
    #6A9955">// First batch job failure detected
})->finally(function ($batch) {
    #6A9955">// Batch finished(success or failure)
})->dispatch();

This pattern is highly effective for heavy data processing, similar to the concepts discussed in Laravel Queues and Redis Lua for Atomic Job Batching.

Hands-on Exercise: The Project Summary Pipeline

In our ongoing project board app, we want to implement a "Project Archival" feature.

1. Create a job that marks the project as "archived" in the database.
2. Create a batch of jobs that archive each individual task within that project.
3. Use a completion callback to trigger a "Project Archived" notification.

Task: Write a service method ArchiveProjectService::execute($project) that uses Bus::batch() to process all tasks. Use the finally hook to log the completion of the archival process.

Common Pitfalls

• Forgetting ShouldQueue: If your jobs don't implement the ShouldQueue interface, they will execute synchronously, blocking your main process regardless of whether you used Bus::chain() or Bus::batch().
• Database Deadlocks: When batching thousands of jobs, the job_batches table can become a bottleneck. Ensure your database is properly indexed, and if you're hitting performance limits, consider using a Redis driver for your queue connection.
• Ignoring Failures: A batch will continue to run even if one job fails, unless you specifically use the allowFailures() method or handle the catch block to cancel the batch. Always define how your application should react to partial failures.

Recap

Job chaining provides a robust way to enforce sequential execution, while batching allows for powerful parallel processing with lifecycle hooks. By offloading these complex patterns to our background workers, we keep our API responsive and our application architecture clean.

Up next: We'll move into testing these asynchronous workflows with Feature Testing Fundamentals.