Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →jpoindexter/structured-output-and-tool-calling
Reference-grade guide to getting reliable structured output and tool/function calls out of LLMs — JSON mode vs constrained/grammar-guided decoding, response_format json_schema and tool-schema coercion, Zod/Pydantic validation at the boundary, the validate→repair→escalate→fallback loop, tool-argument validation, idempotency for safe retries, fallback chains, and evals.
Passed 22 of 22 test cases with the skill, and 21 of 22 without it.
1 fixed21 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
The instructions your model reads when the skill loads.
The moment an LLM's output feeds deterministic code — a parser, a database write, a payment call — "usually valid" is a production incident waiting on a percentage. This skill is about closing that gap: making the structure guaranteed, the contents validated, the execution idempotent, and the failures recoverable.
Core mental model — three independent guarantees, do not conflate them:
Constrained decoding gives you #1 for free and nothing else. You still own #2 and #3.
The single most important distinction in this whole skill:
| Mechanism | What it does | Guarantee | Still need to validate? |
|---|---|---|---|
JSON mode (response_format: {type:"json_object"}) | Biases the model toward JSON via prompt + light logit nudging | Best-effort. Can still truncate, emit wrong types, omit fields | Yes — always |
Constrained / grammar-guided decoding (json_schema strict, GBNF, Outlines) | At every sampling step, masks out tokens that would violate the grammar/schema | Structurally valid by construction — cannot emit a token that breaks the schema | Yes, for semantics — but never for parse/shape |
| Free-text + parse (no mode) | Nothing; you JSON.parse whatever comes out | None. Prose wrappers, markdown fences, commentary | Yes, and brace for failure |
Why constrained decoding eliminates malformed JSON: the decoder maintains a grammar state machine. Before sampling each token, it computes the set of tokens that keep the partial output grammar-valid and sets every other token's logit to -inf. The model literally cannot sample a } where the schema requires more fields, or a string where it requires an integer. There is no "the model usually closes its braces" — closing is the only legal move. This is the difference between guaranteed-valid and best-effort. If you have access to constrained decoding for a schema-shaped task, use it; it deletes an entire class of bugs.
response_format: {type:"json_schema", json_schema:{name, strict:true, schema:{...}}}. With strict:true this is constrained decoding: output is guaranteed to match the JSON Schema (subset: every field required, additionalProperties:false, limited type expressivity). The older {type:"json_object"} is JSON mode — best-effort, validate it.tools with JSON-Schema parameters; strict:true on a function applies the same constrained guarantee to its arguments.json_schema response_format. The stable pattern is tool-schema coercion: define a single tool whose input_schema is your target schema, then force it with tool_choice:{type:"tool", name:"..."}. The model fills the tool's arguments = your structured object. Read it from the tool_use block. Validation still required; coercion biases shape but is not token-masked grammar enforcement.Outlines (regex + JSON-Schema → FSM-masked logits), llama.cpp GBNF grammars, vLLM/SGLang/TGI guided-decoding backends (guided_json, guided_grammar). These are true constrained decoding and work with any JSON Schema or hand-written grammar.GBNF sketch — a grammar IS the contract:
root ::= "{" ws "\"action\"" ws ":" ws action ws "," ws "\"id\"" ws ":" ws integer ws "}"
action ::= "\"create\"" | "\"update\"" | "\"delete\"" # enum, structurally enforced
integer::= "-"? [0-9]+
test cases passed, vs 21 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.