Outlines (v1) API idiom

Contract

Enforces the CURRENT (v1) Outlines public API whenever you write code that uses the outlines library for structured LLM generation — JSON/Pydantic, choice, regex, integer, or free text. The pre-v1 API (outlines.models.transformers("name"), outlines.generate.json(...)) was removed; emit the v1 form below. Apply to any request to "use outlines" / "generate structured output with outlines."

Rules

1. Create the model with a from_<backend> factory — the model constructors were renamed to a

from_ prefix. Use outlines.from_transformers(...), outlines.from_openai(...), outlines.from_vllm(...), outlines.from_llamacpp(...), outlines.from_ollama(...), outlines.from_gemini(...), outlines.from_anthropic(...). Never outlines.models.<backend>(...) — the outlines.models constructor namespace is gone.

• outlines.from_transformers takes two objects: a loaded transformers model AND its tokenizer —

outlines.from_transformers(AutoModelForCausalLM.from_pretrained(name), AutoTokenizer.from_pretrained(name)). It does NOT accept a model-name string.

• outlines.from_openai takes a client object then the model id: outlines.from_openai(openai.OpenAI(), "gpt-4o").

1. Generate by passing an output type — the generate module is gone. Two forms:

• direct call: result = model(prompt, output_type).
• reusable generator: from outlines import Generator → generator = Generator(model, output_type) →

result = generator(prompt). Never outlines.generate.json(...) / .choice(...) / .regex(...) / .text(...) / .integer(...) — the whole outlines.generate module was removed.

1. Express the output type with a Python type or an Outlines type — passed as the second argument,

NOT via a v0 helper:

• JSON object → a pydantic BaseModel subclass, a @dataclass, a TypedDict, or

outlines.types.JsonSchema(schema_string).

• one-of choices → typing.Literal["a", "b", "c"] or an Enum — never a Python list of strings.
• pattern → outlines.types.Regex(r"...") — a raw pattern wrapped in Regex.
• integer / float → the builtins int / float.
• free text → None.

1. The call ALWAYS returns a raw str. Parse it yourself — it is not an already-built object.

For a pydantic model: obj = MyModel.model_validate_json(result). For a JsonSchema: json.loads(result). Do NOT write result.name / iterate result.items as if the return were the parsed instance.

1. Pass sampling and inference options as keyword arguments on the call, not positionally:

model(prompt, output_type, max_new_tokens=256, stop_strings=".", temperature=0.7, top_p=0.9, seed=10). The keywords map to the backend's own inference API.

1. Stream and batch through model/generator methods. Token streaming:

for chunk in model.stream(prompt, output_type): .... Many inputs at once: model.batch([prompt1, prompt2], output_type). Generators expose the same __call__, stream, batch.

Worked examples

Model init (BEFORE = removed v0 → AFTER = v1):

python
# BEFORE
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
# AFTER
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(name),
    AutoTokenizer.from_pretrained(name),
)

JSON / Pydantic:

python
# BEFORE
generator = outlines.generate.json(model, User)
user = generator(prompt)          # v0 returned a User instance
print(user.name)
# AFTER
result = model(prompt, User)      # v1 returns a raw str
user = User.model_validate_json(result)
print(user.name)

Choice:

python
# BEFORE
gen = outlines.generate.choice(model, ["positive", "negative", "neutral"])
# AFTER
from typing import Literal
label = model(prompt, Literal["positive", "negative", "neutral"])

Regex + integer:

python
# BEFORE
phone = outlines.generate.regex(model, r"[0-9]{3}-[0-9]{3}-[0-9]{4}")(prompt)
count = outlines.generate.integer(model)(prompt)
# AFTER
from outlines.types import Regex
phone = model(prompt, Regex(r"[0-9]{3}-[0-9]{3}-[0-9]{4}"))
count = model(prompt, int)

Reusable generator + keyword params:

python
generator = outlines.Generator(model, Product)
result = generator(prompt, max_new_tokens=200, stop_strings=".")
product = Product.model_validate_json(result)

Edge cases

• OpenAI / Gemini / Anthropic backends still go through a from_ factory around the provider client:

outlines.from_openai(openai.OpenAI(), "gpt-4o"), outlines.from_gemini(...). There is no outlines.models.openai(...).

• Raw JSON schema (no Pydantic) → wrap the schema string in outlines.types.JsonSchema(schema_string)

and pass that as the output type; the result is still a str to json.loads.

• Free-text generation → pass output type None (or build Generator(model) with no type); still a str.
• Nested / list models → a single top-level pydantic BaseModel whose fields are nested models or

list[...]; you still pass that one class and parse the returned str once.

Do / Don't

• DO outlines.from_transformers(model_obj, tokenizer_obj). DON'T outlines.models.transformers("name").
• DO pass the output type as the 2nd arg (model(prompt, Schema) / Generator(model, Schema)).

DON'T call outlines.generate.json/choice/regex/integer.

• DO use typing.Literal[...]/Enum for choices. DON'T pass a list of strings.
• DO use outlines.types.Regex(r"..."). DON'T pass a bare pattern to a generate.* helper.
• DO parse the returned str (Model.model_validate_json(result)). DON'T read attributes off the return value.
• DO pass max_new_tokens=, stop_strings=, seed= as keywords. DON'T pass them positionally.

Common mistakes

• Loading a model with outlines.models.transformers("microsoft/Phi-3-...") — removed; use from_transformers

with a model object + tokenizer object.

• Reaching for outlines.generate.json(model, Schema) — the generate module is gone.
• Passing choices as generate.choice(model, ["a","b"]) instead of Literal["a","b"].
• Treating the return value as a parsed Pydantic object (user.name) — v1 returns a str; call

.model_validate_json first.

• Passing inference args positionally (generator(prompt, 256, ".")) instead of max_new_tokens=256, stop_strings=".".

Quick checklist

• ] Model built with outlines.from_<backend>(...) (transformers = model obj + tokenizer obj).
• ] Generation via model(prompt, output_type) or Generator(model, output_type) — no outlines.generate.*.
• ] Output type is a Python/Outlines type: BaseModel/dataclass/JsonSchema, Literal/Enum, Regex, int/float, None.
• ] Return handled as a raw str and parsed (.model_validate_json / json.loads).
• ] Inference options passed as keyword args.