LLM APIs: Prompts, Structured Outputs and Validation

A useful AI feature needs more than a successful prompt. The application must know what it asked for, which model and settings it used, what came back, and whether that response can be accepted.

This lesson builds on Python, HTTP, and JSON fundamentals. You will design a model-response contract for the same fictional document assistant, then test that contract without paying for model calls.

Give the model a bounded task

Start with an application instruction that states the task and evidence boundary: answer the question only from the supplied excerpts; cite supporting excerpt IDs; return insufficient_context when the evidence is inadequate.

Keep those instructions separate from user questions and retrieved documents. A document can contain text that looks like a command. It remains source data, rather than permission to change the application’s rules. The OWASP prompt-injection guidance explains why instructions hidden in external material are a distinct risk.

A useful prompt also defines what not to infer. If the handbook says a refund window is 14 days, the model should not invent exceptions, eligibility rules, or an appeal process. When the question needs absent details, abstention can be the correct result.

Prompt clarity helps, but it is not an enforcement mechanism by itself. The application still decides which documents may be sent and which outputs may be accepted.

Choose a response schema

Our example uses three fields:

{"status":"answered","answer":"The refund window is 14 calendar days.","citations":["refunds-v1"]}

Status is either answered or insufficient_context. Answer is a nonempty string with an application-defined length limit. Citations is a list of unique source IDs.

Where supported, use the provider’s schema-constrained output feature rather than asking for JSON only in prose. OpenAI’s Structured Outputs documentation describes JSON-schema output and supported-schema limitations. Feature availability depends on the model and endpoint. A model being available on an account does not automatically mean it supports every output feature.

A schema can constrain form, but an answer that fits it can still be wrong. “The refund window is 90 days” has the same structure as the correct answer. Semantic support must be checked against the source.

Validate at several layers

First, check transport and provider status. Do not treat an incomplete generation or refusal as a completed answer. Set a timeout so the application does not wait indefinitely.

Second, parse the output. Reject missing fields, unexpected extra fields, wrong types, and unsupported status values. The capstone also bounds answer length and rejects duplicate citations.

Third, validate references. Every citation must identify an excerpt that this request actually retrieved. It is not enough that the ID exists somewhere in the whole database. Citing an inaccessible or unretrieved record crosses the intended evidence boundary.

Finally, assess the claim itself. The cited excerpt may be real but irrelevant. A claim can exaggerate a source or combine facts incorrectly. For consequential uses, include human-reviewed cases and an escalation route rather than assuming citation validation proves truth.

For abstentions, the capstone normalizes the answer to a fixed insufficient-information message. That prevents a response from saying insufficient_context while smuggling an unsupported answer into another field.

Keep provider integration replaceable

The OpenAI Responses API reference documents the request and response shapes used by the optional capstone adapter. The adapter keeps its model ID configurable because model availability and capabilities can change. It does not assume that a particular model name is available to every reader.

The default implementation is a local fixture function. The optional live implementation performs generation from retrieved text. This separation lets you test most application behavior without a network connection, paid request, or nondeterministic model output. Our OpenAI coverage hub collects related provider updates; use the official API documentation for the current contract.

Record whether an experiment used a fixture, mocked HTTP response, or live model. Those are different kinds of evidence. A mocked test establishes that your code handles a chosen response shape; it does not establish that the real model will reliably produce a useful answer.

Errors, retries, and spending

Distinguish a malformed request from a transient provider failure. Retrying an unsupported model or invalid schema usually repeats the same failure. Rate-limited or temporary failures may justify bounded retries, following provider guidance and any Retry-After header. Do not create an unlimited retry loop.

The teaching capstone intentionally makes no automatic retry. Its optional live request has a 20-second client timeout and an output-token limit. Those choices bound parts of the interaction; they are not a guarantee of a fixed monetary charge or provider-side cancellation after a client timeout.

If you add retries, track attempts and total usage per user operation. Compare cost per successful task, rather than treating a low advertised token price as the whole application cost.

Exercise: break the response contract

Write a validation function that accepts an answer and the retrieved source IDs. Start with one good response, then create these cases:

• Answered status with no citation.
• A citation to an ID that was not retrieved.
• A citations string where a list is required.
• An unknown status or a non-string status.
• Insufficient_context with a fabricated answer or supporting citations.

The valid response should pass. Invalid outputs should fail clearly or be normalized according to an explicit abstention policy. Add a mocked refusal and incomplete provider response; neither should become a displayed answer.

You have completed this lesson when the checks run without a model account and you can explain the remaining gap: typed JSON and valid source IDs still do not prove that a claim is supported.

Documentation checked November 8, 2026. The live provider integration is described from primary documentation and tested with mocked responses in the capstone, not paid API calls.