---
name: guidance-api-idiom
source: https://app.decimal.ai/s/guidance-api-idiom@1/SKILL.md
source_sha256: 406646b476a6
---

# guidance (current Python API) idiom

## Contract

Enforces the CURRENT `guidance` (guidance-ai) library Python API whenever you write code that uses
`guidance` for constrained / structured generation. The pre-0.1 handlebars-template API — a
triple-quoted `guidance('''... {{gen 'x'}} ...''')` program, `guidance.llms.OpenAI(...)`, the global
`guidance.llm`, `{{#user}}...{{/user}}` role blocks — was removed. Emit the composition API below for
any request to "use guidance" / "write guidance code."

## Rules

1. **Import the pieces as callables, not a template engine.** Pull the model classes and the
   generation helpers from the package:
   `from guidance import models, gen, select, system, user, assistant, guidance`
   (the model classes also import directly, e.g. `from guidance.models import Transformers`).

2. **Build a model with a capitalized class under `guidance.models`.**
   - `lm = models.Transformers("microsoft/Phi-4-mini-instruct")` — a local Hugging Face checkpoint by id.
   - `lm = models.LlamaCpp("/path/to/model.gguf", n_ctx=4096)` — a local GGUF file.
   - `lm = models.OpenAI("gpt-4o")` — a remote endpoint.
   Never `guidance.llm = guidance.llms.OpenAI(...)` and never `guidance.llms.<backend>(...)`: the
   `guidance.llms` namespace and the module-global `guidance.llm` were removed.

3. **Compose the program with the `+` / `+=` operator on the (immutable) model object.** Add plain
   strings for fixed text and `gen(...)` / `select(...)` for generated spans:
   `lm += "Question: " + q + "\nAnswer: " + gen("answer")`. A model object is immutable, so `lm += x`
   rebinds `lm` to a new object. Do NOT pass a triple-quoted string containing `{{gen ...}}` /
   `{{select ...}}` to `guidance(...)` — that handlebars program form no longer exists.

4. **Generate with the `gen()` function; the capture name is the FIRST argument.**
   `gen("city", max_tokens=10, stop="\n")`, `gen("zip", regex=r"\d{5}")`. Options are keyword
   arguments: `max_tokens`, `regex`, `stop`, `stop_regex`, `temperature`. Never `{{gen 'city'}}`.

5. **Constrain to a fixed set with `select()`; the options list is the FIRST positional argument.**
   `select(["GET", "POST", "PUT", "DELETE"], name="method")`. Never `{{select 'method' options=...}}`,
   and never a bare `gen()` you hope lands inside the set.

6. **Read a captured value by indexing the resulting model object: `lm["city"]`.** The key is the
   name you passed to `gen` / `select`. There is no separate compile-then-execute step, so never build a
   `program` and read `program["city"]` after calling it.

7. **Use the role context managers for chat models — not handlebars role blocks.**
   `with system(): lm += "..."`, `with user(): lm += "..."`, `with assistant(): lm += gen("reply")`.
   Never `{{#system}}...{{/system}}` / `{{#user}}...{{/user}}` / `{{#assistant}}...{{/assistant}}`.

8. **Package reusable pieces with the `@guidance` decorator.** The decorated function takes `lm` as its
   first parameter, adds to it, and returns it; then it composes like any other span:
   `@guidance` \ `def rate(lm, item): lm += f"{item}: " + select(["low", "high"], name="r"); return lm`,
   used as `lm += rate("latency")` (or `lm = rate(lm, "latency")`). Never save a handlebars template
   string as the "reusable" unit.

9. **For JSON constrained to a schema, use `guidance.json(...)`.** Pass a pydantic `BaseModel` (or a
   JSON schema) via `schema=`: `lm += guidance.json("out", schema=Address)`. Never hand-roll a
   `{{gen 'json'}}` block or a free `gen()` and hope it parses.

## Worked examples

Model construction (BEFORE = removed handlebars/llms form → AFTER = current):
```python
# BEFORE
import guidance
guidance.llm = guidance.llms.OpenAI("gpt-4")
# AFTER
from guidance import models
lm = models.OpenAI("gpt-4o")
```

Generate + capture (BEFORE → AFTER):
```python
# BEFORE
program = guidance('''The animal is a {{gen 'animal' max_tokens=5}}''')
out = program()
print(out["animal"])
# AFTER
from guidance import models, gen
lm = models.Transformers("microsoft/Phi-4-mini-instruct")
lm += "The animal is a " + gen("animal", max_tokens=5)
print(lm["animal"])
```

Fixed choice (BEFORE → AFTER):
```python
# BEFORE
program = guidance('''Signal: {{select 'color' options=lights}}''')
# AFTER
from guidance import select
lm += "Signal: " + select(["red", "yellow", "green"], name="color")
print(lm["color"])
```

Chat roles (BEFORE → AFTER):
```python
# BEFORE
program = guidance('''{{#system}}You are a tutor.{{/system}}{{#user}}What is 6*7?{{/user}}{{#assistant}}{{gen 'a'}}{{/assistant}}''')
# AFTER
from guidance import system, user, assistant, gen
with system():
    lm += "You are a tutor."
with user():
    lm += "What is 6*7?"
with assistant():
    lm += gen("a")
```

Reusable component + schema JSON:
```python
from guidance import guidance, gen, select, json
from pydantic import BaseModel

@guidance
def yes_no(lm, question):
    lm += question + " " + select(["yes", "no"], name="verdict")
    return lm

class Book(BaseModel):
    title: str
    year: int

lm += yes_no("Is the sky blue?")
lm += guidance.json("book", schema=Book)
```

## Edge cases

- **Regex-constrained span** → keyword on `gen`: `gen("date", regex=r"\d{4}-\d{2}-\d{2}")`. Never a
  removed `{{gen 'date' pattern=...}}` directive.
- **Stop condition** → `gen("line", stop="\n")` (or `stop_regex=...`), passed as a keyword — not a
  positional trailing string.
- **Repeated captures in a loop** → give `select`/`gen` `list_append=True` and read the list back from
  `lm[name]`; or capture into distinct names. Still composed with `+=`, never a `{{#geneach}}` block.
- **Anthropic / other backends** → still a class under `guidance.models` where supported; the
  constrained primitives (`gen`, `select`, `guidance.json`) require a backend that exposes token control
  (local `Transformers` / `LlamaCpp`), so prefer those for `regex` / `select` guarantees.
- **f-strings for fixed text are fine** — interpolate Python values into the string you add
  (`lm += f"User: {name}\n"`), but generated spans always come from `gen`/`select`, never string
  formatting.

## Do / Don't

- DO `models.Transformers("id")` / `models.OpenAI("gpt-4o")`. DON'T `guidance.llms.OpenAI(...)` or set `guidance.llm`.
- DO compose with `lm += "text" + gen("x")`. DON'T pass a `{{gen}}` template to `guidance('''...''')`.
- DO call `gen("x", max_tokens=…, regex=…)`. DON'T write `{{gen 'x'}}`.
- DO call `select(["a","b"], name="x")` (options first). DON'T write `{{select 'x' options=...}}`.
- DO read captures via `lm["x"]`. DON'T execute a `program(...)` and index it.
- DO open roles with `with system()/user()/assistant():`. DON'T use `{{#system}}...{{/system}}`.
- DO wrap reusable logic in `@guidance def f(lm, ...): ...; return lm`. DON'T store a template string.
- DO constrain JSON with `guidance.json("x", schema=Model)`. DON'T free-generate and hope it parses.

## Common mistakes

- Setting a global `guidance.llm = guidance.llms.OpenAI(...)` — the global model and `guidance.llms` were removed.
- Wrapping the whole prompt in `guidance('''... {{gen 'x'}} ...''')` and calling the returned program — the handlebars program is gone; compose with `+=`.
- Emitting `{{gen 'x'}}` / `{{select 'x' options=...}}` / `{{#user}}...{{/user}}` directives anywhere.
- Passing options to `select` by keyword (`select(name="x", options=[...])`) instead of options-first `select([...], name="x")`.
- Reading a result from `program["x"]` after execution instead of `lm["x"]` on the composed model.
- Using a plain `gen()` for JSON instead of `guidance.json(..., schema=Model)`.

## Quick checklist

- [ ] Model built with `models.Transformers/LlamaCpp/OpenAI(...)` — no `guidance.llms` / global `guidance.llm`.
- [ ] Program composed with `lm += "text" + gen(...)/select(...)` — no `guidance('''...{{...}}...''')`.
- [ ] `gen("name", …)` with the capture name first; `select(["…"], name="…")` options-first.
- [ ] Captures read via `lm["name"]`.
- [ ] Chat roles via `with system()/user()/assistant():`.
- [ ] Reusable spans via `@guidance def f(lm, …): …; return lm`; schema JSON via `guidance.json(..., schema=…)`.