Distributed Tracing: Implementing OpenTelemetry in Laravel

Previously in this course, we explored custom telemetry design to track business-level metrics. While those metrics tell you what is happening, Distributed Tracing tells you where and why it’s happening, especially when a request hops across multiple services, queues, or internal modules.

When your system grows into a modular monolith or a collection of microservices, standard logging often fails to capture the "full story" of a request. You need a way to correlate operations across process boundaries.

Understanding Distributed Tracing from First Principles

At its core, Tracing is the process of recording the journey of a single request. Every request is assigned a unique Trace ID. As that request touches different parts of your system—a Controller, a Service, a database query, or an external API call—each operation is recorded as a Span.

A Span captures:

• Operation name (e.g., OrderProcessingAction)
• Start and end timestamps
• Metadata (tags, attributes, or logs)
• Parent-child relationships (to build the tree structure)

The OpenTelemetry Standard

Rather than relying on proprietary vendor agents, we use OpenTelemetry (OTel). It is the industry-standard framework for collecting and exporting telemetry data. By instrumenting your Laravel application with OTel, you decouple your observability data from your storage backend (like Jaeger, Honeycomb, or AWS X-Ray).

Implementing OpenTelemetry in Laravel

To implement tracing, we need to instrument the application to generate spans. In a production Laravel environment, we typically use the open-telemetry/opentelemetry-php library.

1. Installation

Install the necessary OTel packages via Composer:

Bash
composer require open-telemetry/opentelemetry-php-all

2. The Instrumentation Strategy

You shouldn't manually start spans in every controller. Instead, leverage middleware to capture the incoming request and use an Action decorator or a service provider to auto-instrument your domain logic.

Here is a simplified example of a middleware that starts a trace for every incoming request:

PHP
namespace App\Http\Middleware;

use OpenTelemetry\API\Trace\TracerInterface;
use Closure;

class TraceRequest
{
    public function __construct(protected TracerInterface $tracer) {}

    public function handle($request, Closure $next)
    {
        $span = $this->tracer->spanBuilder($request->path())->startSpan();
        $scope = $span->activate();

        try {
            return $next($request);
        } finally {
            $span->end();
            $scope->detach();
        }
    }
}

3. Propagating Context

The real power of Observability comes when you pass the Trace ID to downstream services. If you call an external API or push to a queue, you must inject the trace context into the outgoing headers.

For queues, as discussed in our Laravel OpenTelemetry Distributed Tracing: Async Queue Context Propagation guide, you should extract the trace context from the job payload to ensure the worker continues the parent trace rather than starting a new one.

Visualizing Request Paths

Once spans are flowing to your collector, you can visualize the latency of every step.

Sequence diagram: participant User; participant Gateway; participant OrderModule; participant Database; User → Gateway: POST /orders; Gateway → OrderModule: Dispatch Action; OrderModule → Database: Insert Order; Database → OrderModule: Success; OrderModule → Gateway: Response; Gateway → User: 201 Created

By inspecting the trace in a tool like Jaeger or Grafana Tempo, you can identify "long poles"—the specific operations taking the most time—and pinpoint exactly where a failure occurred during a distributed transaction.

Hands-on Exercise: Tracing an Action

Your task is to instrument one of your Action classes.

1. Inject the TracerInterface into your ProcessPaymentAction.
2. Wrap the core logic in a span: $span = $this->tracer->spanBuilder('process_payment')->startSpan();.
3. Add an attribute to the span: $span->setAttribute('payment.amount', $dto->amount);.
4. View the trace in your local development collector.

Common Pitfalls

• Over-instrumentation: Creating too many spans can lead to "span explosion," increasing CPU usage and storage costs. Focus on high-level boundaries (Actions, API calls, DB queries).
• Context Leaks: Always use finally blocks to close spans. If a span isn't closed due to an exception, you will leak memory and corrupt your trace trees.
• Ignoring Async Boundaries: As we noted in Distributed tracing for asynchronous microservices: A practical guide, forgetting to propagate headers through Redis or SQS will result in fragmented traces.

Recap

Tracing is not just for debugging; it is a critical component of modern Observability. By implementing OpenTelemetry, you gain deep insights into request lifecycles, allowing you to optimize performance and reduce Debugging time in complex architectures. Always prioritize context propagation across your service boundaries to maintain a unified view of your system.

Up next: Profiling PHP Execution — where we will dive deeper into analyzing execution traces using Xdebug and Blackfire.