guidance (current Python API) idiom

Contract

Enforces the CURRENT guidance (guidance-ai) library Python API whenever you write code that uses guidance for constrained / structured generation. The pre-0.1 handlebars-template API — a triple-quoted guidance('''... {{gen 'x'}} ...''') program, guidance.llms.OpenAI(...), the global guidance.llm, {{#user}}...{{/user}} role blocks — was removed. Emit the composition API below for any request to "use guidance" / "write guidance code."

Rules

1. Import the pieces as callables, not a template engine. Pull the model classes and the

generation helpers from the package: from guidance import models, gen, select, system, user, assistant, guidance (the model classes also import directly, e.g. from guidance.models import Transformers).

1. Build a model with a capitalized class under guidance.models.

• lm = models.Transformers("microsoft/Phi-4-mini-instruct") — a local Hugging Face checkpoint by id.
• lm = models.LlamaCpp("/path/to/model.gguf", n_ctx=4096) — a local GGUF file.
• lm = models.OpenAI("gpt-4o") — a remote endpoint.

Never guidance.llm = guidance.llms.OpenAI(...) and never guidance.llms.<backend>(...): the guidance.llms namespace and the module-global guidance.llm were removed.

1. Compose the program with the + / += operator on the (immutable) model object. Add plain

strings for fixed text and gen(...) / select(...) for generated spans: lm += "Question: " + q + "\nAnswer: " + gen("answer"). A model object is immutable, so lm += x rebinds lm to a new object. Do NOT pass a triple-quoted string containing {{gen ...}} / {{select ...}} to guidance(...) — that handlebars program form no longer exists.

1. Generate with the gen() function; the capture name is the FIRST argument.

gen("city", max_tokens=10, stop="\n"), gen("zip", regex=r"\d{5}"). Options are keyword arguments: max_tokens, regex, stop, stop_regex, temperature. Never {{gen 'city'}}.

1. Constrain to a fixed set with select(); the options list is the FIRST positional argument.

select(["GET", "POST", "PUT", "DELETE"], name="method"). Never {{select 'method' options=...}}, and never a bare gen() you hope lands inside the set.

1. Read a captured value by indexing the resulting model object: lm["city"]. The key is the

name you passed to gen / select. There is no separate compile-then-execute step, so never build a program and read program["city"] after calling it.

1. Use the role context managers for chat models — not handlebars role blocks.

with system(): lm += "...", with user(): lm += "...", with assistant(): lm += gen("reply"). Never {{#system}}...{{/system}} / {{#user}}...{{/user}} / {{#assistant}}...{{/assistant}}.

1. Package reusable pieces with the @guidance decorator. The decorated function takes lm as its

first parameter, adds to it, and returns it; then it composes like any other span: @guidance \ def rate(lm, item): lm += f"{item}: " + select(["low", "high"], name="r"); return lm, used as lm += rate("latency") (or lm = rate(lm, "latency")). Never save a handlebars template string as the "reusable" unit.

1. For JSON constrained to a schema, use guidance.json(...). Pass a pydantic BaseModel (or a

JSON schema) via schema=: lm += guidance.json("out", schema=Address). Never hand-roll a {{gen 'json'}} block or a free gen() and hope it parses.

Worked examples

Model construction (BEFORE = removed handlebars/llms form → AFTER = current):

python
# BEFORE
import guidance
guidance.llm = guidance.llms.OpenAI("gpt-4")
# AFTER
from guidance import models
lm = models.OpenAI("gpt-4o")

Generate + capture (BEFORE → AFTER):

python
# BEFORE
program = guidance('''The animal is a {{gen 'animal' max_tokens=5}}''')
out = program()
print(out["animal"])
# AFTER
from guidance import models, gen
lm = models.Transformers("microsoft/Phi-4-mini-instruct")
lm += "The animal is a " + gen("animal", max_tokens=5)
print(lm["animal"])

Fixed choice (BEFORE → AFTER):

python
# BEFORE
program = guidance('''Signal: {{select 'color' options=lights}}''')
# AFTER
from guidance import select
lm += "Signal: " + select(["red", "yellow", "green"], name="color")
print(lm["color"])

Chat roles (BEFORE → AFTER):

python
# BEFORE
program = guidance('''{{#system}}You are a tutor.{{/system}}{{#user}}What is 6*7?{{/user}}{{#assistant}}{{gen 'a'}}{{/assistant}}''')
# AFTER
from guidance import system, user, assistant, gen
with system():
    lm += "You are a tutor."
with user():
    lm += "What is 6*7?"
with assistant():
    lm += gen("a")

Reusable component + schema JSON:

python
from guidance import guidance, gen, select, json
from pydantic import BaseModel

@guidance
def yes_no(lm, question):
    lm += question + " " + select(["yes", "no"], name="verdict")
    return lm

class Book(BaseModel):
    title: str
    year: int

lm += yes_no("Is the sky blue?")
lm += guidance.json("book", schema=Book)

Edge cases

• Regex-constrained span → keyword on gen: gen("date", regex=r"\d{4}-\d{2}-\d{2}"). Never a

removed {{gen 'date' pattern=...}} directive.

• Stop condition → gen("line", stop="\n") (or stop_regex=...), passed as a keyword — not a

positional trailing string.

• Repeated captures in a loop → give select/gen list_append=True and read the list back from

lm[name]; or capture into distinct names. Still composed with +=, never a {{#geneach}} block.

• Anthropic / other backends → still a class under guidance.models where supported; the

constrained primitives (gen, select, guidance.json) require a backend that exposes token control (local Transformers / LlamaCpp), so prefer those for regex / select guarantees.

• f-strings for fixed text are fine — interpolate Python values into the string you add

(lm += f"User: {name}\n"), but generated spans always come from gen/select, never string formatting.

Do / Don't

• DO models.Transformers("id") / models.OpenAI("gpt-4o"). DON'T guidance.llms.OpenAI(...) or set guidance.llm.
• DO compose with lm += "text" + gen("x"). DON'T pass a {{gen}} template to guidance('''...''').
• DO call gen("x", max_tokens=…, regex=…). DON'T write {{gen 'x'}}.
• DO call select(["a","b"], name="x") (options first). DON'T write {{select 'x' options=...}}.
• DO read captures via lm["x"]. DON'T execute a program(...) and index it.
• DO open roles with with system()/user()/assistant():. DON'T use {{#system}}...{{/system}}.
• DO wrap reusable logic in @guidance def f(lm, ...): ...; return lm. DON'T store a template string.
• DO constrain JSON with guidance.json("x", schema=Model). DON'T free-generate and hope it parses.

Common mistakes

• Setting a global guidance.llm = guidance.llms.OpenAI(...) — the global model and guidance.llms were removed.
• Wrapping the whole prompt in guidance('''... {{gen 'x'}} ...''') and calling the returned program — the handlebars program is gone; compose with +=.
• Emitting {{gen 'x'}} / {{select 'x' options=...}} / {{#user}}...{{/user}} directives anywhere.
• Passing options to select by keyword (select(name="x", options=[...])) instead of options-first select([...], name="x").
• Reading a result from program["x"] after execution instead of lm["x"] on the composed model.
• Using a plain gen() for JSON instead of guidance.json(..., schema=Model).

Quick checklist

• ] Model built with models.Transformers/LlamaCpp/OpenAI(...) — no guidance.llms / global guidance.llm.
• ] Program composed with lm += "text" + gen(...)/select(...) — no guidance('''...{{...}}...''').
• ] gen("name", …) with the capture name first; select(["…"], name="…") options-first.
• ] Captures read via lm["name"].
• ] Chat roles via with system()/user()/assistant():.
• ] Reusable spans via @guidance def f(lm, …): …; return lm; schema JSON via guidance.json(..., schema=…).