What Can We Trust in a Model’s Structured Output?

A predictable response format helps software read the answer. It still has to check the proposed action.

A model returns a refund request with every required field: payment ID, amount, currency, and reason. The response parses correctly. Before the application sends money back, it still needs to establish whether this is the right payment and whether the refund is allowed.

Suppose the customer has two similar charges. The model selects the wrong one but produces perfectly valid JSON. A schema validator can accept the response because the fields have the expected types. The application must compare the proposed payment with the customer’s records and the evidence for a duplicate charge.

Structured outputs let a team specify the shape of a model’s response. Constrained generation restricts the forms or values the model can produce, depending on the implementation. These techniques reduce parsing errors and make responses easier for software to consume. They do not establish that a field is true or that the proposed operation is permitted.

This is Post 17 in The Abstraction Shift: How Software Keeps Moving Up.

Why software needs more than a request for JSON

The first model integration often returns prose. A user asks a question, the model answers, and the application displays the result.

The next integration asks the model to produce something another component can consume. The application may want a list of extracted fields, a classification, a user-interface description, a search plan, or the arguments for a tool call.

The simplest request is usually: “Return JSON.”

That instruction improves the appearance of the response. It does not create a contract.

The model may omit a field, invent an enum value, add an unexpected property, wrap the object in a sentence, or return a value with the wrong type. The application then parses the response, retries when parsing fails, and gradually accumulates prompts, regular expressions, repair functions, and special cases.

The system has recreated a fragile parser around a component that can produce many plausible forms.

Structured outputs address part of that problem by specifying the expected schema. A constrained decoder, grammar, parser, or provider-side structured-output feature can narrow the possible response space. A response can then cross a syntactic boundary with much greater reliability.

The production problem continues after parsing. A schema can say that amount is a number and currency is a string. It cannot, by itself, know whether the amount belongs to the authenticated account, whether the currency matches the payment record, whether a refund is allowed, or whether the payment system actually processed it.

Current structured-output APIs make this distinction visible. They distinguish valid JSON from adherence to a supplied schema, support structured responses separately from tool calls, and still require applications to handle refusals, incomplete output, unsupported schema features, and domain-specific checks. The format boundary becomes stronger. The surrounding software still owns the meaning of the operation.

The checks software already makes before acting

Software has crossed data boundaries through contracts for a long time.

A function signature states the arguments a function accepts and the value it returns. An interface definition language describes messages that can cross a process or service boundary. A parser turns bytes into a data structure. A type checker rejects some invalid programs before they run. An API validates a request before it reaches a handler. A database constraint protects an invariant at the point where state changes.

The familiar path looks something like this:

input -> parse -> validate shape -> validate meaning -> authorize -> execute -> verify outcome

Each step has a different responsibility. A parser can identify a malformed date. It cannot decide whether the date is acceptable for a particular business process. A type checker can establish that a value is an integer. It cannot establish that the integer identifies a resource the caller may modify.

Model-mediated software keeps the path but changes the producer at the front. Instead of a deterministic program constructing every value from a known control flow, a model interprets context and proposes a value that should fit the contract.

That proposal can be useful precisely because the path is open to interpretation. A support request may use different words for the same operation. A research task may require a different set of fields for each source. A user-interface request may describe a goal rather than a fixed component tree.

The model adds interpretation. The contract keeps interpretation from becoming an unexamined side effect.

Structured output moves the response closer to an interface contract. The proposal still needs semantic validation and authority checks before execution.

What Is a Structured Output?

A structured output gives the response a declared shape, such as an object with a payment identifier, an amount, and a currency. JSON Schema is one way to describe those fields, their types, and the values allowed. The application can then check the response without trying to extract an operation from prose.

Constrained generation limits the forms the model can produce while it generates the response. Different implementations use grammars, decoding restrictions, or validation and retries, so the supported guarantees need to be checked for the chosen provider.

The format is only one part of an agreement between components. A refund operation also needs rules about which payment may be refunded, who may request it, and which result means money moved. The following table separates those checks so a passing format check cannot stand in for the rest.

Layer	Question	Typical owner
Shape	Can the value be parsed, and does it have the required fields and types?	Parser and schema validator
Semantics	Do the fields represent a coherent claim or operation?	Domain validator and application logic
Policy	Is the proposed behavior allowed by the governing rules?	Policy engine and application owner
Authority	May this identity perform this operation on this resource?	Authentication, authorization, and capability boundary
Execution	Can the operation run safely with the required limits and idempotency?	Runtime and capability implementation
Outcome	Did the intended state change actually occur?	Source-of-truth system and verifier

Structured output mainly strengthens the first layer. It can help the model express the later layers by including evidence references, reasons, uncertainty, or requested approval. It cannot replace the later layers.

This distinction also separates response formatting from tool calling. A response schema can organize the assistant’s answer for a user interface. A tool schema describes a capability invocation that may cross into an external system. The second path carries authority and side effects, so the runtime needs stronger validation and auditability.

From a model response to a proposal the application can check

The important change is not that models can emit JSON. Models could emit JSON before structured-output features existed.

The change is that model output is becoming a deliberate intermediate representation between human intent and deterministic execution.

A compiler accepts source text, parses it into a representation, checks rules, and produces a result that another runtime can execute. A model-mediated application can use a similar boundary, with an important difference: the model’s intermediate representation is inferred from context and may be uncertain, incomplete, or wrong.

The model therefore acts like an interpreter of intent and a generator of proposals. The runtime acts like the authority that decides what the proposal means operationally.

This boundary works best when the schema makes uncertainty and branching explicit: a clarification_required state on a proposal, an unknown category on a classifier, approval markers on the steps of a plan, or a research result that keeps observed facts, hypotheses, and unresolved conflicts apart.

The schema should give the runtime a useful state machine rather than force every situation into a successful-looking object.

The schema is one checkpoint in a longer path from intent to an authorized and verified state change.

The boundary also changes how teams design interfaces. A conventional interface often starts with the operation the program already knows how to perform. A model-mediated interface starts with the set of interpretations the model may propose and the set of actions the runtime is prepared to govern.

That makes the schema a design artifact. It should answer:

• What does each field mean?
• Which fields are evidence, and which are decisions?
• Which values are authoritative and which are suggestions?
• How does the proposal represent uncertainty, refusal, or missing information?
• Which fields are safe to expose to the model?
• Which fields must be derived or checked by the runtime?
• What version owns the contract, and what happens when it changes?

The schema is part of the application boundary. Treating it as a prompt fragment hides that responsibility.

Where the familiar comparisons stop helping

Structured output resembles a type system, but the analogy stops at several important points.

A valid shape can carry a false claim

A schema can require a string called customer_id. It cannot establish that the string identifies the customer discussed in the request. A schema can require a number called refund_amount. It cannot establish that the amount is supported by the payment record.

Structural validity is evidence about representation. It is not evidence that the representation is true.

Constrained generation is not semantic validation

A decoder can constrain an enum to refund, clarify, or deny. The model can still choose refund for a case that should be denied. Domain validation must inspect current state, authoritative records, policy, and identity after the proposal exists.

A schema does not define authority

The model may produce a perfectly valid request to delete a record or transfer funds. The schema can describe the request. It cannot grant permission to make it. Authority belongs to the authenticated caller, the runtime, and the capability boundary.

A schema can encode the wrong abstraction

An overly broad schema turns every uncertain case into a plausible success. An overly narrow schema forces meaningful variation into arbitrary strings or null values. The contract must represent the states the application can actually govern.

Provider support is part of the contract

Structured-output implementations support different subsets of schema features, limits, refusal behavior, and streaming semantics. A schema that works on one model or provider may fail at configuration time or behave differently after routing changes. Capability checks and route policy belong beside the schema, not in an undocumented assumption.

The proposal is not the outcome

A model can propose “send the invoice” and a tool can return “accepted.” The message may still be queued, rejected later, delivered to the wrong destination, or never reach the recipient. The application needs an outcome model that distinguishes proposal, authorization, execution, provider acknowledgement, and verified completion.

Checking a proposal before it reaches the payment system

A robust structured-output path separates proposal from authority:

1. Assemble context. Select the user request, relevant state, source records, instructions, tools, and provenance that the model may use.
2. Declare the contract. Define fields, types, allowed values, uncertainty states, error branches, version, and descriptions. Keep authoritative identifiers and sensitive values outside the proposal when the runtime can derive them.
3. Generate within the boundary. Use structured output, constrained decoding, a grammar, or a provider-specific function interface when it improves adherence. Record the model, route, schema version, and refusal or truncation state.
4. Parse and validate shape. Reject extra fields, missing required values, invalid types, unsupported combinations, and incomplete output. Treat a refusal or interrupted response as an explicit branch.
5. Validate meaning. Resolve identifiers, check current state, compare amounts, verify evidence, enforce invariants, and distinguish a fact from a model-generated hypothesis.
6. Apply policy and authority. Check user scope, agent identity, capability permissions, approval requirements, data boundaries, and risk limits.
7. Execute narrowly. Send only the validated operation to a capability with its own authorization, timeout, idempotency, and retry rules.
8. Verify the outcome. Read the source of truth, reconcile the requested state change, record the evidence, and present the user with the verified status.

The model may participate in steps 1 and 3. It can help interpret context, select a candidate operation, and explain a result. The runtime owns steps 4 through 8 wherever the path can affect authoritative state.

A useful contract records more than fields. It states what the proposal means, who may act on it, how failure is represented, and what proves completion.

The contract should also describe its failure vocabulary. Returning an empty string for an unknown customer, a zero amount for a failed extraction, or an empty array for unavailable evidence collapses different states into values that look valid. Explicit unknown, needs_clarification, refused, not_found, and validation_failed branches preserve the difference.

Schema versioning matters for the same reason API versioning matters. Additive fields can still change model behavior, descriptions can influence selection, enum changes can alter control flow, and a provider’s supported subset can change the set of deployable contracts. Version the schema, test migrations, and record the version in traces and stored proposals.

Following the example through the system

Consider a support agent that receives this request:

Find the duplicate charge and refund it if the account and refund policy allow.

The model may need to identify the account, find recent payment records, compare amounts and merchants, distinguish a duplicate from two legitimate purchases, and decide whether it has enough evidence to propose a refund.

A useful proposal contract might contain:

Field	Purpose	Runtime treatment
operation	refund, clarify, or deny	Validate against the allowed state transitions
payment_id	Candidate payment to inspect	Resolve against the authenticated account
amount and currency	Proposed refund amount	Compare with the authoritative payment record and policy limit
reason	Model’s explanation of the duplicate claim	Preserve as an explanation, not as proof
evidence_ids	Records that support the proposal	Check provenance, freshness, and account scope
missing_information	Information needed before a decision	Route to clarification rather than guessing
approval_required	A proposed indication that review may be needed	Recompute from policy; do not trust the model’s flag

The runtime then follows a governed path:

1. Authenticate the user and establish the support agent’s account scope.
2. Retrieve payment records and label their authority and freshness.
3. Ask the model for a structured proposal or an explicit clarification state.
4. Validate the proposal’s structure and allowed enum values.
5. Resolve payment_id and compare the amount and currency with the payment system.
6. Check duplicate-charge evidence, refund limits, identity requirements, and approval policy.
7. Call a narrow refund capability with an idempotency key.
8. Query the payment system to verify whether the refund committed.
9. Record the proposal, validation decisions, authorization, execution result, and verified state.
10. Tell the user whether the refund completed, needs approval, or remains uncertain.

The same schema supports several outcomes, but the schema does not choose the outcome. The model can propose refund. The runtime decides whether the proposal corresponds to a real duplicate, whether the identity may act, and whether the money actually moved.

If the payment tool times out, the runtime must not treat a second model proposal as a safe retry. It needs an idempotency key and a way to query the authoritative payment state. If the state is unknown, the correct response may be “I am checking whether the refund completed,” followed by a later reconciliation step.

This is where structured outputs connect to guardrails and evals. A live guardrail controls the proposal path. An evaluation tests whether the model produced the right branch, whether invalid proposals were rejected, whether authorization preceded execution, whether the tool ran once, and whether the final response matched the verified payment state.

What the team needs to design and operate

Architecture

Schemas become shared boundary artifacts. Store them with the code or contract registry that owns the capability. Define the validator, semantic checks, authority boundary, error branches, versioning policy, and outcome verifier together.

Keep proposals separate from committed domain state. A RefundProposal should never be stored or read as a Refund, and the same holds for a CreateOrderRequest and an Order, or a ResearchFinding and an authoritative record. The names and storage model should preserve that distinction.

Engineering

Test the contract at several levels:

• schema-valid examples with wrong meaning;
• missing, null, stale, and conflicting values;
• prompt-injected or untrusted evidence;
• unsupported provider features and refusal paths;
• policy and authorization failures;
• duplicate execution, timeouts, and ambiguous outcomes;
• schema migrations and route changes.

Use evals to measure model behavior, but use deterministic validators and source-of-truth checks for the properties that can be checked deterministically.

Product and UX

Structured contracts create better product states. The application can ask for the missing identifier, show a proposed action before approval, explain that evidence conflicts, or display that a task is still being reconciled. These states are more useful than forcing every turn to become a confident answer.

The user should see the right level of detail for the consequence. A low-risk classification may remain invisible. A refund, deployment, or external message may need a visible summary of target, scope, amount, and approval state before execution.

Operations and governance

Monitor contract adherence, semantic rejection, refusal, clarification, approval, execution, and outcome rates separately. A high schema-adherence rate can coexist with poor task outcomes. A high completion rate can coexist with unsafe authorization if the checks are missing.

Assign an owner to each contract. Record schema version, model route, input and evidence scope, validation decisions, side effects, and outcome evidence. When a model or provider changes, the contract and its evaluation suite should make the behavior change visible.

Failure Modes

Valid shape, wrong meaning

A proposal passes schema validation while naming the wrong record or asserting an unsupported fact. Run semantic checks against authoritative state.

JSON mode mistaken for contract enforcement

Valid JSON can still omit required fields or use the wrong enum. Use schema-constrained output where supported, then retain application-side validation.

Constraint illusion

The provider accepts a schema but supports only a subset of its features, or a fallback route silently uses a weaker format. Make capability and route compatibility explicit.

Null as a universal escape hatch

Optional fields and nullable values can make every uncertain case appear valid. Model uncertainty as named states with clear runtime handling.

Proposal committed as state

The application writes the model’s proposal directly into an authoritative table. Store proposals separately and require domain validation and authorization before commitment.

Contract drift

Prompt changes, schema descriptions, model updates, routing changes, and tool revisions alter the effective interface. Version the contract and evaluate the complete path.

Completion claimed from a tool response

A tool acknowledgement becomes a success message before the source-of-truth state is checked. Separate requested, authorized, accepted, committed, and verified states.

Reviewing the output contract

Before a structured proposal crosses into deterministic software, complete this contract card:

Question	Record
What decision or capability does this proposal support?	Purpose and boundary
Which fields are model-proposed, runtime-derived, or authoritative?	Ownership per field
Which values and combinations are structurally valid?	Schema and version
Which invariants require domain or source-of-truth checks?	Semantic validators
Who may act, on which resource, under which policy?	Identity and authority
How are refusal, clarification, ambiguity, and failure represented?	Explicit branches
What makes the operation safe to retry?	Idempotency and retry policy
What evidence proves the requested result occurred?	Outcome verification
Who owns the contract and how is change evaluated?	Owner, tests, and rollout plan

The card keeps the schema connected to the system that gives it meaning. Whatever format a future system uses, it will still need to resolve the payment, apply policy, establish permission, and verify the result, which is why the schema belongs with the operation’s validators and tests.

Sources

• Structured model outputs, OpenAI API: documents schema-constrained responses, function calling, response formats, supported schema subsets, strict adherence, refusals, incomplete output, and the distinction between JSON mode and schema adherence.
• What is JSON Schema?, JSON Schema: explains structural validation and why sufficiently complex systems need a second semantic validation phase.
• JSON Schema specification: defines the current JSON Schema Core and Validation specifications and the 2020-12 dialect.
• The 2026-07-28 MCP specification release candidate: documents the movement of MCP tool input and output schemas toward full JSON Schema 2020-12 and the role of structured tool content.
• MCP Tasks extension: shows how structured tool results and task handles can represent long-running, asynchronous work beyond one blocking call.