LLM Provider Setup

Overview

This skill is the canonical procedure for adding a new LLM provider, model, or inference profile to a Vellum assistant. Follow the steps in order — each step's output feeds the next, and the final live-call verification is mandatory. Never skip ahead by writing raw config JSON.

Run the steps strictly sequentially — one command per step, and read its output before running the next. Never batch create → verify → activate into a single turn: creation can fail validation, verification can fail on a wrong model id or missing credential, and activation must not happen until verification passed. Each step's output is the gate for the next one.

Step 0 — Check what's already available (avoid collecting keys unnecessarily)

Managed (platform-credentialed) routing may already cover the user's need — no API key required:

assistant inference providers list          # provider entries; `vellum` is the platform-managed route
assistant inference providers default       # default provider + availability status
assistant inference profiles list           # effective profiles: managed + user, with availability

Managed first. If the user is signed in to Vellum and the model they asked for is served by the managed route, build the profile on it — --provider vellum --model <model-id>, no --connection, no credential, nothing to prompt for. Skip Steps 1 and 2 entirely and go to Step 3; there is no key to collect. If the model turns out not to be managed-routable, profile creation says so explicitly (Step 4) — only then fall back to key collection.

Collect an API key only when there is genuinely no managed option: the user is not signed in to Vellum, the model is not served by the managed route, or the user explicitly wants to use their own key.

Step 1 — Reuse an existing key, or securely collect a new one

Before prompting the user for anything, check whether a suitable key is already stored:

assistant credentials list

If a credential for the target provider exists, reuse it — reference it by vault path in Step 2 and skip the prompt. Only collect a new key when none exists (or the user explicitly wants to replace it).

Never ask for secrets in chat, and never send the user to the Settings page for this. The key must not enter the conversation, and the collection happens inline in the current conversation — the secure prompt renders a masked input right where the user already is:

assistant credentials prompt --service <provider> --field api_key \
  --label "<Provider> API Key" --placeholder "sk-..."

Exit code 0 = stored; exit code 130 = the user dismissed the prompt (a valid choice, not an error — ask whether they want to try again or stop). Any other non-zero exit is a real error.

Step 2 — Create the provider connection