LLM guardrails: Validating Function-Calling Output with Pydantic

During a recent sprint, our team spent three days debugging a pipeline that was failing because an LLM agent decided to return a "null" field instead of the expected integer for a price calculation. We had assumed the model would "just know" the schema, but in production, assumptions are just bugs waiting to happen. If you're building AI agents, you need to treat their output as untrusted user input.

Implementing LLM guardrails is the only way to move from a brittle prototype to a system that doesn't crash when the model gets creative.

Why Pydantic validation is non-negotiable

When you use function calling, you're asking the model to act as a structured data generator. However, LLMs are probabilistic engines, not compilers. Even with strict system prompts, they can occasionally omit required keys or hallucinate types.

We first tried simple regex matching to clean up the JSON strings coming back from the API. It worked for about 20% of cases before we hit a nested object that broke our logic. We then switched to Structured output with Pydantic: A Guide to Reliable LLM Parsing, which gave us the type safety we needed. By defining a schema, you create a contract that the agent must respect.

Implementing Pydantic for function-calling validation

To get started, you'll need the pydantic library (v2.x is recommended for better performance) and your choice of LLM provider SDK.

Here is how we define a robust schema for a search agent:

PYTHON
from pydantic import BaseModel, Field, field_validator
from typing import List

class SearchQuery(BaseModel):
    query: str = Field(..., description="The user's search intent")
    category: str = Field(..., description="The product category")
    limit: int = Field(default=5, ge=1, le=20)

    @field_validator(CE9178">'query')
    @classmethod
    def check_length(cls, v: str):
        if len(v) < 3:
            raise ValueError(CE9178">'Query too short')
        return v

The field_validator is key here. It allows you to enforce AI agent security by rejecting inputs that are too short, clearly nonsensical, or potentially malicious before they ever hit your database.

The validation flow

Once you have your model, you need to bridge the gap between the LLM's response and your Python object. Most modern frameworks like LangChain or Instructor handle this, but understanding the raw flow is essential for debugging.

Flow diagram: User Input → LLM Agent; LLM Agent → LLM Response; LLM Response → Raw JSON Pydantic Validation; Pydantic Validation → Invalid Error/Retry; Pydantic Validation → Valid Business Logic

Moving beyond basic schema checks

When you're dealing with LLM guardrails for production: Input validation and output filtering, you realize that schema validation is only half the battle. You also need to handle retries.

If the validation fails, you shouldn't just crash. Instead, catch the ValidationError and feed it back to the model as a system message. This creates a self-correcting loop.

Practical tips for structured data extraction

When refining your structured data extraction pipelines, keep these two things in mind:

1. Keep schemas flat: Deeply nested Pydantic models can sometimes confuse smaller models (like GPT-4o-mini or Haiku). If you’re getting consistent errors, flatten the model.
2. Use strict descriptions: The description field in Pydantic is actually a prompt instruction for the LLM. Be as specific as possible about what the field expects.

If you are building more complex systems, consider how this fits into your wider architecture. We often combine this with LLM entity extraction for knowledge graph construction: A practical guide to ensure that the nodes we extract are cleaned and validated before they hit the graph database.

Frequently Asked Questions

Q: Does Pydantic validation add significant latency? A: In our testing, the overhead of Pydantic validation is roughly 2-5ms, which is negligible compared to the 500ms+ latency of the LLM inference itself.

Q: What happens if the LLM refuses to follow the schema? A: Use a "retry mechanism" where you pass the Pydantic error message back to the LLM. It usually corrects the output on the second attempt about 90% of the time.

Q: Should I use this for every function call? A: Yes. If the output is being used to trigger a database write or an external API call, you should always enforce a schema to maintain system integrity.

Ultimately, guardrails aren't about stopping the AI from being creative; they're about ensuring the AI speaks the language your code understands. I'm still experimenting with how to handle "soft" failures—cases where the model is 90% correct but the schema is strict—but for now, Pydantic remains our standard for production stability.