---
name: outlines-v1-api
source: https://app.decimal.ai/s/outlines-v1-api@1/SKILL.md
source_sha256: 924895e1fd6c
---

# Outlines (v1) API idiom

## Contract

Enforces the CURRENT (v1) Outlines public API whenever you write code that uses the `outlines`
library for structured LLM generation — JSON/Pydantic, choice, regex, integer, or free text. The pre-v1
API (`outlines.models.transformers("name")`, `outlines.generate.json(...)`) was removed; emit the v1
form below. Apply to any request to "use outlines" / "generate structured output with outlines."

## Rules

1. **Create the model with a `from_<backend>` factory** — the model constructors were renamed to a
   `from_` prefix. Use `outlines.from_transformers(...)`, `outlines.from_openai(...)`,
   `outlines.from_vllm(...)`, `outlines.from_llamacpp(...)`, `outlines.from_ollama(...)`,
   `outlines.from_gemini(...)`, `outlines.from_anthropic(...)`. Never `outlines.models.<backend>(...)`
   — the `outlines.models` constructor namespace is gone.
   - `outlines.from_transformers` takes **two objects**: a loaded transformers model AND its tokenizer —
     `outlines.from_transformers(AutoModelForCausalLM.from_pretrained(name), AutoTokenizer.from_pretrained(name))`.
     It does NOT accept a model-name string.
   - `outlines.from_openai` takes a client object then the model id: `outlines.from_openai(openai.OpenAI(), "gpt-4o")`.

2. **Generate by passing an output type — the `generate` module is gone.** Two forms:
   - direct call: `result = model(prompt, output_type)`.
   - reusable generator: `from outlines import Generator` → `generator = Generator(model, output_type)` →
     `result = generator(prompt)`.
   Never `outlines.generate.json(...)` / `.choice(...)` / `.regex(...)` / `.text(...)` / `.integer(...)`
   — the whole `outlines.generate` module was removed.

3. **Express the output type with a Python type or an Outlines type** — passed as the second argument,
   NOT via a v0 helper:
   - JSON object → a pydantic `BaseModel` subclass, a `@dataclass`, a `TypedDict`, or
     `outlines.types.JsonSchema(schema_string)`.
   - one-of choices → `typing.Literal["a", "b", "c"]` or an `Enum` — never a Python list of strings.
   - pattern → `outlines.types.Regex(r"...")` — a raw pattern wrapped in `Regex`.
   - integer / float → the builtins `int` / `float`.
   - free text → `None`.

4. **The call ALWAYS returns a raw `str`.** Parse it yourself — it is not an already-built object.
   For a pydantic model: `obj = MyModel.model_validate_json(result)`. For a JsonSchema: `json.loads(result)`.
   Do NOT write `result.name` / iterate `result.items` as if the return were the parsed instance.

5. **Pass sampling and inference options as keyword arguments** on the call, not positionally:
   `model(prompt, output_type, max_new_tokens=256, stop_strings=".", temperature=0.7, top_p=0.9, seed=10)`.
   The keywords map to the backend's own inference API.

6. **Stream and batch through model/generator methods.** Token streaming:
   `for chunk in model.stream(prompt, output_type): ...`. Many inputs at once:
   `model.batch([prompt1, prompt2], output_type)`. Generators expose the same `__call__`, `stream`, `batch`.

## Worked examples

Model init (BEFORE = removed v0 → AFTER = v1):
```python
# BEFORE
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# AFTER
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(name),
    AutoTokenizer.from_pretrained(name),
)
```

JSON / Pydantic:
```python
# BEFORE
generator = outlines.generate.json(model, User)
user = generator(prompt)          # v0 returned a User instance
print(user.name)
# AFTER
result = model(prompt, User)      # v1 returns a raw str
user = User.model_validate_json(result)
print(user.name)
```

Choice:
```python
# BEFORE
gen = outlines.generate.choice(model, ["positive", "negative", "neutral"])
# AFTER
from typing import Literal
label = model(prompt, Literal["positive", "negative", "neutral"])
```

Regex + integer:
```python
# BEFORE
phone = outlines.generate.regex(model, r"[0-9]{3}-[0-9]{3}-[0-9]{4}")(prompt)
count = outlines.generate.integer(model)(prompt)
# AFTER
from outlines.types import Regex
phone = model(prompt, Regex(r"[0-9]{3}-[0-9]{3}-[0-9]{4}"))
count = model(prompt, int)
```

Reusable generator + keyword params:
```python
generator = outlines.Generator(model, Product)
result = generator(prompt, max_new_tokens=200, stop_strings=".")
product = Product.model_validate_json(result)
```

## Edge cases

- **OpenAI / Gemini / Anthropic backends** still go through a `from_` factory around the provider client:
  `outlines.from_openai(openai.OpenAI(), "gpt-4o")`, `outlines.from_gemini(...)`. There is no
  `outlines.models.openai(...)`.
- **Raw JSON schema (no Pydantic)** → wrap the schema *string* in `outlines.types.JsonSchema(schema_string)`
  and pass that as the output type; the result is still a str to `json.loads`.
- **Free-text generation** → pass output type `None` (or build `Generator(model)` with no type); still a str.
- **Nested / list models** → a single top-level pydantic `BaseModel` whose fields are nested models or
  `list[...]`; you still pass that one class and parse the returned str once.

## Do / Don't

- DO `outlines.from_transformers(model_obj, tokenizer_obj)`. DON'T `outlines.models.transformers("name")`.
- DO pass the output type as the 2nd arg (`model(prompt, Schema)` / `Generator(model, Schema)`).
  DON'T call `outlines.generate.json/choice/regex/integer`.
- DO use `typing.Literal[...]`/`Enum` for choices. DON'T pass a list of strings.
- DO use `outlines.types.Regex(r"...")`. DON'T pass a bare pattern to a `generate.*` helper.
- DO parse the returned str (`Model.model_validate_json(result)`). DON'T read attributes off the return value.
- DO pass `max_new_tokens=`, `stop_strings=`, `seed=` as keywords. DON'T pass them positionally.

## Common mistakes

- Loading a model with `outlines.models.transformers("microsoft/Phi-3-...")` — removed; use `from_transformers`
  with a model object + tokenizer object.
- Reaching for `outlines.generate.json(model, Schema)` — the `generate` module is gone.
- Passing choices as `generate.choice(model, ["a","b"])` instead of `Literal["a","b"]`.
- Treating the return value as a parsed Pydantic object (`user.name`) — v1 returns a `str`; call
  `.model_validate_json` first.
- Passing inference args positionally (`generator(prompt, 256, ".")`) instead of `max_new_tokens=256, stop_strings="."`.

## Quick checklist

- [ ] Model built with `outlines.from_<backend>(...)` (transformers = model obj + tokenizer obj).
- [ ] Generation via `model(prompt, output_type)` or `Generator(model, output_type)` — no `outlines.generate.*`.
- [ ] Output type is a Python/Outlines type: BaseModel/dataclass/JsonSchema, Literal/Enum, Regex, int/float, None.
- [ ] Return handled as a raw `str` and parsed (`.model_validate_json` / `json.loads`).
- [ ] Inference options passed as keyword args.