Testing Structured Outputs from an LLM: A Practical Reliability Guide for AI Engineers

When an LLM returns JSON that looks correct, it is tempting to treat the job as done. But most production failures do not show up during generation. They show up two steps downstream, when a missing field breaks a database write or an invented status value silently misroutes a support ticket. This is exactly where API testing and structured output validation become a discipline in their own right, rather than an afterthought bolted onto prompt engineering.. Provider-native features have made structured outputs far more reliable than the free-form text LLMs produced even a year ago, but reliable is not the same as guaranteed. A response can be perfectly parseable, fully schema-valid, and still be wrong, pointing at the wrong record, contradicting itself, or inventing a value the model was never given.

This guide walks through a layered, practical approach to testing structured outputs from any LLM: verifying completion status, parsing safely, validating against a schema, and running the semantic checks that catch errors a schema alone can never see. Whether you are using OpenAI’s native Structured Outputs feature or building your own validation layer on top of another provider, the same core principle holds throughout this guide: parseable does not mean valid, and valid does not mean correct.

• What Are Structured Outputs?
• Why Testing Matters
• Pipeline
• Retries
• Test Suite
• Quality Metrics
• Best Practices
• Common Mistakes
• Troubleshooting
• Tools
• Limitations
• Conclusion

What Are Structured Outputs, and How Should They Be Tested?

Structured outputs testing is the process of verifying that an LLM response satisfies a machine-readable output contract. The contract normally includes three levels:

• Syntactic validity: Is the response valid JSON?
• Structural validity: Does the parsed object conform to the expected schema?
• Semantic validity: Are the values correct, internally consistent, and grounded in the input?

JSON itself defines objects, arrays, strings, numbers, booleans, and null values, but valid JSON does not impose application-specific requirements such as mandatory fields or allowed status values. Those constraints belong in a schema or application validation layer.

For example, the following is valid JSON:

{
  "priority": "extremely_high",
  "confidence": 4.8
}

It may still be invalid for an application that allows only low, medium, high, or urgent priorities and requires confidence to fall between 0 and 1.

Why Testing Structured Outputs Matters

Structured outputs are often passed directly into databases, APIs, workflow engines, user interfaces, or automated decision systems. A malformed or misleading value can therefore produce an application failure even when the response looks plausible to a person.

Common consequences include:

• A missing identifier causing a database write to fail.
• An unsupported enum value breaking downstream routing.
• A string being returned where a number is expected.
• A truncated response causing JSON parsing to fail.
• A schema-valid but incorrect value triggering the wrong business action.
• Blind retries increasing latency, token usage, and rate-limit pressure.
• A changed model or prompt introducing regressions that were not detected during development.

Provider-native structured outputs features reduce some of these risks. For example, OpenAI Structured Outputs can constrain supported models to a supplied JSON Schema, unlike basic JSON mode, which guarantees JSON syntax but not schema adherence. However, the documentation also warns that a model may produce schema-compliant hallucinations when the source input cannot reasonably satisfy the schema. Schema enforcement therefore does not eliminate the need for semantic checks.

How Does a Reliable Structured Outputs Pipeline Work?

A production pipeline should validate the response in a fixed order:

• Inspect the API result. Check whether generation completed, failed, was refused, or stopped because of an output limit.
• Extract the intended output. Do not assume every response contains a normal assistant message.
• Parse the JSON. Reject malformed syntax, surrounding prose, Markdown fences, or incomplete objects unless the integration explicitly supports them.
• Validate the schema. Check required properties, types, enums, ranges, patterns, array rules, and additional properties.
• Run semantic checks. Compare values with the source input and enforce cross-field business rules.
• Classify the failure. Distinguish transport, truncation, parsing, schema, semantic, refusal, and policy failures.
• Apply a targeted recovery action. Retry only when a retry can reasonably change the outcome.
• Record the result. Store failure category, attempt count, model configuration, latency, token use, and validator messages.

The ordering matters. A response marked incomplete should not be treated as an ordinary JSON parse failure, and a schema-valid object should not be accepted before domain rules have been evaluated.

Parsing, Schema Validation, and Semantic Validation Compared

Each layer should produce a distinct error type. Collapsing every failure into “invalid JSON” makes debugging, retry selection, and quality measurement unnecessarily difficult.

Step 1: Define a Strict JSON Schema

Consider an LLM that converts support tickets into a triage record. A valid output should contain the original ticket ID, a priority from a controlled set, a supported category, a human-review decision, a concise summary, and a confidence score between zero and one.

TRIAGE_SCHEMA = {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
        "ticket_id": {
            "type": "string",
            "pattern": r"^T-\d{4}$"
        },
        "priority": {
            "type": "string",
            "enum": ["low", "medium", "high", "urgent"]
        },
        "category": {
            "type": "string",
            "enum": ["billing", "technical", "account", "other"]
        },
        "requires_human": {
            "type": "boolean"
        },
        "summary": {
            "type": "string",
            "minLength": 10,
            "maxLength": 300
        },
        "confidence": {
            "type": "number",
            "minimum": 0,
            "maximum": 1
        }
    },
    "required": [
        "ticket_id", "priority", "category",
        "requires_human", "summary", "confidence"
    ],
    "additionalProperties": False
}

JSON Schema uses keywords such as required, enum, minimum, maximum, and additionalProperties to express structural constraints. The enum keyword restricts a value to a fixed set, while additionalProperties: false rejects fields that were not defined in the object schema.

What Should Be Required?

Mark a field as required when the downstream application cannot safely or unambiguously continue without it. Good candidates include:

• Record identifiers.
• Action or routing decisions.
• Units for measurements.
• Currency codes for monetary values.
• Evidence or reason fields for high-impact decisions.
• Schema or payload version identifiers.

Avoid making a field optional merely because the model might omit it. Optionality should represent a legitimate domain state, not unreliable generation. Where a value is genuinely unknown, model that state deliberately using a nullable field, an explicit unknown enum value, a separate availability flag, or a discriminated union with different required fields.

Step 2: Validate the Schema Itself

A malformed schema can produce confusing results or inconsistent validator behavior. Validate the schema during application startup or continuous integration rather than discovering the problem during a live request.

from jsonschema import Draft202012Validator

Draft202012Validator.check_schema(TRIAGE_SCHEMA)
validator = Draft202012Validator(TRIAGE_SCHEMA)

The Python jsonschema library provides validator classes for supported schema drafts and a check_schema method for validating a schema against its meta-schema. Keep the schema version explicit otherwise, different libraries or services may interpret keywords according to different JSON Schema drafts.

Step 3: Detect Truncation Before Parsing

Do not rely only on a parser error such as “unexpected end of input.” Inspect the provider’s response status first. Depending on the API, truncation indicators may include:

• An incomplete response status.
• An incomplete reason such as max_output_tokens.
• A legacy finish reason such as length.
• A streaming connection ending before the final completion event.
• A provider-specific maximum-token stop reason.

OpenAI’s Responses API can return an incomplete status with an incomplete reason when generation reaches the output-token limit or context boundary. Its documentation recommends allocating sufficient output space or adjusting the request when this occurs.

def completion_error(status: str, incomplete_reason: str | None) -> str | None:
    if status == "completed":
        return None
    if status == "incomplete":
        return f"Generation incomplete: {incomplete_reason or 'unknown reason'}"
    return f"Generation did not complete successfully: {status}"

Only parse the payload after the provider reports a completed response.

How Should a Truncated Response Be Retried?

Do not resend the identical request automatically. Change the condition that caused the truncation by doing one or more of the following:

• Increase the permitted output budget.
• Reduce the expected array size.
• Divide the task into batches.
• Remove unnecessary explanatory fields.
• Shorten the source context.
• Request pagination or continuation through an explicit protocol.
• Replace free-form text fields with bounded alternatives.

For large extraction jobs, returning 500 items in one object is usually less reliable than requesting 25 bounded items per page with a cursor or source offset.

Step 4: Parse JSON Without Attempting Unsafe Repair

After completion has been confirmed, parse the response with the standard parser for the application language.

import json
from json import JSONDecodeError
from typing import Any

def parse_json(raw_text: str) -> tuple[Any | None, list[str]]:
    try:
        return json.loads(raw_text), []
    except JSONDecodeError as exc:
        return None, [
            f"JSON parse error at line {exc.lineno}, "
            f"column {exc.colno}: {exc.msg}"
        ]

Avoid silently “fixing” malformed JSON through broad string replacement. Naive repair logic can change values, remove meaningful characters, or transform an unsafe output into an apparently valid object.

For example, globally replacing single quotes with double quotes could corrupt apostrophes inside legitimate text. Removing all text before the first { could also hide an important refusal or warning.

A safer repair strategy is:

• Retain the original response.
• Record the exact parser error.
• Make at most one targeted repair request when appropriate.
• Re-run every validation layer on the new output.
• Never treat repaired content as trusted merely because it parses.

Step 5: Validate Required Fields and Invalid Values

Once the payload has been parsed, run the schema validator and collect all available errors rather than stopping at the first one.

from typing import Any

def schema_errors(data: Any) -> list[str]:
    errors = sorted(
        validator.iter_errors(data),
        key=lambda error: list(error.absolute_path)
    )
    formatted: list[str] = []
    for error in errors:
        path = ".".join(str(part) for part in error.absolute_path)
        location = path or "$"
        formatted.append(f"{location}: {error.message}")
    return formatted

Collecting all validation errors produces better diagnostics and allows a repair prompt to address several related problems in a single retry.

The unexpected-field test is particularly important. An LLM may invent a seemingly useful property that downstream code was never designed to interpret.

Step 6: Run Semantic Checks After Parsing

Semantic validation tests meaning rather than representation. A payload can satisfy every schema constraint while still being wrong:

{
  "ticket_id": "T-9999",
  "priority": "urgent",
  "category": "billing",
  "requires_human": false,
  "summary": "The customer reports a duplicate charge.",
  "confidence": 0.94
}

The object is structurally valid, but it may violate two business rules: the returned ticket ID must match the source ticket, and every urgent ticket must require human review.

from typing import Any

def semantic_errors(
    data: dict[str, Any],
    source_ticket_id: str
) -> list[str]:
    errors: list[str] = []

    if data["ticket_id"] != source_ticket_id:
        errors.append("ticket_id does not match the source record")

    if data["priority"] == "urgent" and not data["requires_human"]:
        errors.append("urgent tickets must require human review")

    if data["confidence"] < 0.60 and not data["requires_human"]:
        errors.append("low-confidence classifications must require human review")

    if not data["summary"].strip():
        errors.append("summary must contain non-whitespace text")

    return errors

What Should Semantic Checks Verify?

The exact checks depend on the workflow, but common categories include:

• Source grounding: Returned IDs exactly match source IDs; names, dates, quantities, and monetary values appear in the source; extracted quotations are exact substrings when required; every classification includes supporting evidence.
• Cross-field consistency: end_date is not earlier than start_date; subtotal + tax = total within tolerance; a rejected request does not include an approval action.
• Business rules: Currency and country combinations are supported; a refund does not exceed the original transaction; a user cannot approve their own high-value request.
• Safety constraints: Generated database filters are tenant-scoped; URLs use an approved scheme and domain; file paths remain within an allowed directory.
• Task completeness: Every source record has a corresponding output record; no source record is duplicated; array ordering matches the requested rule.

Semantic checks should be deterministic whenever possible. Use another model as a judge only for criteria that cannot be expressed reliably in code, and evaluate that judge against human-reviewed examples before trusting it.

Complete Python Validation Pipeline for Structured Outputs

The following example combines completion checks, parsing, schema validation, and semantic validation into one pipeline for testing structured outputs end to end.

from __future__ import annotations

import json
from dataclasses import dataclass
from enum import Enum
from json import JSONDecodeError
from typing import Any

from jsonschema import Draft202012Validator


class FailureKind(str, Enum):
    INCOMPLETE = "incomplete"
    PARSE = "parse"
    SCHEMA = "schema"
    SEMANTIC = "semantic"


@dataclass(frozen=True)
class ValidationResult:
    accepted: bool
    data: dict[str, Any] | None
    failure_kind: FailureKind | None
    errors: list[str]


Draft202012Validator.check_schema(TRIAGE_SCHEMA)
VALIDATOR = Draft202012Validator(TRIAGE_SCHEMA)


def validate_llm_output(
    raw_text: str,
    *,
    response_status: str,
    incomplete_reason: str | None,
    source_ticket_id: str,
) -> ValidationResult:
    if response_status != "completed":
        return ValidationResult(
            accepted=False,
            data=None,
            failure_kind=FailureKind.INCOMPLETE,
            errors=[
                "Response did not complete: "
                f"{incomplete_reason or response_status}"
            ],
        )

    try:
        parsed: Any = json.loads(raw_text)
    except JSONDecodeError as exc:
        return ValidationResult(
            accepted=False,
            data=None,
            failure_kind=FailureKind.PARSE,
            errors=[f"Line {exc.lineno}, column {exc.colno}: {exc.msg}"],
        )

    schema_failures = sorted(
        VALIDATOR.iter_errors(parsed),
        key=lambda error: list(error.absolute_path),
    )

    if schema_failures:
        errors: list[str] = []
        for failure in schema_failures:
            path = ".".join(str(part) for part in failure.absolute_path)
            errors.append(f"{path or '$'}: {failure.message}")
        return ValidationResult(
            accepted=False,
            data=None,
            failure_kind=FailureKind.SCHEMA,
            errors=errors,
        )

    semantic_failures = semantic_errors(parsed, source_ticket_id=source_ticket_id)

    if semantic_failures:
        return ValidationResult(
            accepted=False,
            data=parsed,
            failure_kind=FailureKind.SEMANTIC,
            errors=semantic_failures,
        )

    return ValidationResult(
        accepted=True,
        data=parsed,
        failure_kind=None,
        errors=[],
    )

How Should Retries Be Designed?

Retries should be based on failure category rather than a single catch-all rule.

OpenAI’s current rate-limit guidance recommends honoring Retry-After when present and otherwise using exponential backoff with random jitter and a maximum retry count. It also notes that unsuccessful requests consume rate-limit capacity, so immediate repeated requests can make the problem worse.

Use Targeted Retry Prompts

A useful schema-repair prompt contains:

• The original task.
• The schema or relevant constraints.
• The previous output.
• Exact validator errors.
• An instruction to return a complete replacement object.
• A warning not to add commentary or Markdown.

Your previous JSON response failed validation.

Validation errors:
- $.summary: 'summary' is a required property
- $.priority: 'critical' is not an allowed value

Return a complete replacement object.
Allowed priority values: low, medium, high, urgent.
Do not return a patch, explanation, or Markdown.

Do not ask the model to “fix the JSON” without supplying the error. The model may alter valid fields unnecessarily or repeat the same failure.

Bound Retries

A typical policy might allow: two or three transport retries, one truncation retry after changing the request, one schema-repair retry, one semantic-repair retry for a recoverable rule, and no unchanged retry for a refusal or unsupported task. The exact limits should be based on error frequency, latency objectives, request cost, and the consequences of an incorrect result.

Practical Test Suite for Structured Outputs

A reliable evaluation dataset should include both normal and adversarial inputs.

import json
import pytest

VALID_OBJECT = {
    "ticket_id": "T-1042",
    "priority": "high",
    "category": "billing",
    "requires_human": True,
    "summary": "The customer reports a duplicate charge.",
    "confidence": 0.91,
}

@pytest.mark.parametrize(
    ("payload", "expected_failure"),
    [
        (VALID_OBJECT, None),
        (
            {key: value for key, value in VALID_OBJECT.items() if key != "summary"},
            FailureKind.SCHEMA,
        ),
        (
            {**VALID_OBJECT, "priority": "critical"},
            FailureKind.SCHEMA,
        ),
        (
            {**VALID_OBJECT, "confidence": 1.2},
            FailureKind.SCHEMA,
        ),
        (
            {**VALID_OBJECT, "ticket_id": "T-9999"},
            FailureKind.SEMANTIC,
        ),
        (
            {**VALID_OBJECT, "priority": "urgent", "requires_human": False},
            FailureKind.SEMANTIC,
        ),
    ],
)
def test_structured_output(payload, expected_failure):
    result = validate_llm_output(
        json.dumps(payload),
        response_status="completed",
        incomplete_reason=None,
        source_ticket_id="T-1042",
    )
    assert result.failure_kind == expected_failure
    assert result.accepted is (expected_failure is None)


def test_truncated_response():
    result = validate_llm_output(
        '{"ticket_id": "T-1042", "priority": "high"',
        response_status="incomplete",
        incomplete_reason="max_output_tokens",
        source_ticket_id="T-1042",
    )
    assert result.failure_kind == FailureKind.INCOMPLETE
    assert result.accepted is False

Unit tests verify the validator, not the model. Model evaluation requires repeatedly calling the configured LLM across a representative dataset and measuring the resulting pass rates.

How Should Structured Outputs Quality Be Measured?

Track each validation stage separately.

Recommended metrics

• Completion rate — completed responses / total requests
• JSON parse rate — parseable completed responses / completed responses
• Schema pass rate — schema-valid responses / parseable responses
• Semantic pass rate — semantically valid responses / schema-valid responses
• First-attempt acceptance rate — accepted outputs without retry / total requests
• Final acceptance rate — accepted outputs after permitted retries / total requests

Also track: truncation rate, missing-field rate by field, invalid-enum rate by property, unexpected-property rate, semantic failure rate by rule, average attempts per accepted output, refusal and policy-block rates, median and 95th-percentile latency, token usage and cost per accepted output, human escalation rate, and regression rate by prompt, model, and schema version.

Do not report only the final success rate. A system that succeeds after three retries may still be too expensive or slow for production.

Evaluations should run whenever the prompt, model, schema, tool configuration, parsing code, or semantic rules change. Current OpenAI evaluation guidance describes evals as a way to test model outputs against defined style and content criteria, particularly when changing models or application configurations — these evaluation metrics matter as much for structured outputs as they do for open-ended generation.

Best Practices for Testing Structured Outputs

Prefer Native Schema-Constrained Structured Outputs

Use provider-native structured outputs or strict function schemas when the selected model supports them. They reduce malformed responses and many basic schema failures. Continue validating application-side — provider support may cover only a subset of JSON Schema, and semantic correctness remains the application’s responsibility.

Keep Schemas Narrow

Include only fields required by the workflow. Every optional explanatory property creates another opportunity for ambiguity, verbosity, or truncation. Prefer a compact object like {"action": "escalate", "reason_code": "payment_dispute"} over an object containing several long, loosely defined narrative fields when downstream code needs only an action and reason.

Close Objects Deliberately

Use additionalProperties: false when unexpected fields must be rejected. For public or versioned contracts, consider whether strict closure could make future schema evolution harder. A version field or explicit extension object may provide controlled flexibility.

Make Nullability Explicit

Do not assume that an optional property and a nullable property mean the same thing. These represent different states: {}, {"value": null}, and {"value": ""}. Define which states are valid and test each one.

Validate Formats Deliberately

JSON Schema’s format keyword is not automatically enforced by every validator. In Python’s jsonschema implementation, a format checker must be supplied when format assertions are required; otherwise, formats may be treated as informational. For critical dates, emails, identifiers, and URLs, confirm that the selected validator actively checks the relevant format or implement an application-level validator.

Separate Extraction From Decision-Making

Where risk is high, use one stage to extract grounded facts and another deterministic stage to calculate the action. For example: the model extracts invoice amount, payment status, and dispute reason; schema validation verifies the fields; source-grounding checks verify the extracted facts; application code determines refund eligibility. This limits the number of business decisions delegated to probabilistic output.

Preserve the Original Response

Store the raw output with request or trace ID, model identifier, prompt version, schema version, completion status, validation errors, retry history, and accepted normalized output. Redact or encrypt sensitive data according to the application’s privacy requirements.

Test Edge Cases, Not Only Normal Examples

Production failures often occur around empty input, extremely long input, multilingual text, duplicate records, conflicting evidence, invalid dates, very large or very small numbers, escaped quotes and newlines, prompt-injection attempts embedded in source documents, and inputs for which no valid answer exists. The schema and prompt should define how the model represents uncertainty and unsupported cases.

Common Mistakes When Testing Structured Outputs

Troubleshooting Structured Outputs From LLMs

Why does the JSON parser report an unexpected end of input?

Likely cause: Output truncation, an interrupted stream, or a genuinely malformed response.

How to verify: First inspect the provider’s completion status or stop reason.

Solution: When the response reached an output-token limit, increase the output budget or reduce the requested payload. Do not treat a truncated fragment as a normal schema-repair case.

Why are required fields missing even though the prompt lists them?

Likely cause: A prompt instruction is not equivalent to schema enforcement.

Solution: Use a structured outputs feature or function schema where available, mark the properties as required in JSON Schema, and retain application-side validation. For unsupported models, return the validator’s missing-property errors in one targeted retry.

Why does the output pass schema validation but contain the wrong answer?

Likely cause: JSON Schema validates representation and declared constraints, not grounding or truth.

Solution: Compare identifiers, dates, totals, quotations, classifications, and actions with authoritative source data. Apply deterministic business rules and route uncertain high-impact outputs to human review.

Why does a date pass validation even though it is malformed?

Likely cause: The validator may not be enforcing the JSON Schema format keyword.

Solution: Verify whether format checking is enabled and whether the required format is supported. For critical date logic, parse the value with the application’s date library and run checks such as valid calendar date, timezone requirement, and start-before-end.

Why do automatic retries make performance worse?

Likely cause: The application may be retrying permanent or deterministic failures.

Solution: Limit retries, add backoff for transient errors, and change the prompt, schema, output budget, or task size when correcting generation failures.

Why does the model invent values when information is missing?

Likely cause: The schema may require a field without defining a valid unknown state.

Solution: Add explicit handling for insufficient evidence, such as {"status": "insufficient_information", "missing_fields": ["transaction_date"]}, or use a discriminated union that defines separate success and insufficient-information payloads.

Tools and Implementation Options

Keep one canonical contract where possible. Generating unrelated schemas separately for the provider, API documentation, and application model can create drift.

Limitations and Risks of Structured Outputs

Schema support differs by provider

A provider may implement only a subset of JSON Schema. Validate the schema against the provider before deployment and avoid assuming that a locally valid Draft 2020-12 schema can be used unchanged by every model API.

Schema validation cannot prove factual correctness

A perfectly valid object can contain invented names, incorrect totals, unsupported classifications, or unsafe actions. High-impact systems need source verification, deterministic rules, authoritative lookups, or human review.

Strict schemas can hide uncertainty

When every property is required and no unknown state exists, the model may be pushed toward fabricating a value. Design schemas that let the system represent missing evidence honestly.

Retries affect cost and latency

Every generation attempt consumes time and resources. A high final success rate can conceal a poor first-attempt success rate and an uneconomical retry loop.

Large payloads are vulnerable to truncation

Long arrays, verbose evidence fields, and deeply nested objects consume output capacity. Use bounded arrays, pagination, batching, and concise reason codes for large extraction workloads.

Semantic rules require maintenance

Business rules change. Version semantic validators alongside schemas and prompts, and include both versions in logs and evaluation reports.

Conclusion

Reliable structured outputs from an LLM require a layered contract. Start by requesting schema-constrained output where available, but do not stop there. Check whether generation completed, parse the JSON strictly, validate every required field and allowed value, apply deterministic semantic rules, and classify failures before retrying.

The most important principle is that parseable does not mean valid, and valid does not mean correct. Treat those as separate gates, measure each gate independently, and make human review an explicit outcome when evidence is incomplete or the decision is high impact.

Need Help Testing Your LLM Structured Outputs? Let's Talk.

Schedule a Consultation

Frequently Asked Questions