Monthly totals

Overview

This skill walks through analyzing and reducing LLM spend on a Vellum assistant. There are three layers:

1. Provider connections — named auth configs (e.g. anthropic-managed, my-personal-key)
2. Model profiles — named presets (provider + model + effort + thinking + contextWindow). Four managed defaults, with UI labels. Note that the keys do not track the labels: read the key, not the name, when pinning a call site.
   • balanced → Balanced (the general agent-loop profile)
   • quality-optimized → Quality (the expensive escalation profile)
   • cost-optimized → Cost (the cheap utility/background profile, and the one to pin for spend reduction)
   • latency-optimized → Speed (the low time-to-first-token profile, used by live voice; faster but not cheaper than Cost)
3. Call-site profile pins (llm.callSites.<id>.profile) — optional per-task overrides of the shipped defaults.

The concrete model behind each managed profile depends on the install: platform-managed installs and BYOK installs resolve different providers/models, and the catalog changes over time. Never assume which model a profile maps to — read assistant config get llm.profiles and the usage breakdown by model to see what actually ran.

How model selection works — read this before diagnosing

Every LLM call resolves exactly one winning profile through a strict first-usable-wins chain. Profiles never merge with each other:

1. Per-conversation / per-run override — the user's /model pick, an open assistant inference session, or a schedule's pinned profile
2. llm.activeProfile — applies to mainAgent (the chat loop) only; it IS the user's chat-model selection and outranks any llm.callSites.mainAgent pin
3. llm.callSites.<site>.profile — explicit per-site pin
4. The call site's shipped default intent, resolved through llm.defaultProvider
5. balanced intent (final anchor)

A rung only wins if its profile exists, is enabled, and carries its own provider + model; otherwise resolution silently falls to the next rung.

Consequences that change how you diagnose cost:

• A missing or empty llm.callSites block is healthy, not a red flag. Every call site ships with a sensible default intent: the agent loop and quality-sensitive sites (mainAgent, subagentSpawn, compactionAgent, callAgent, patternScan, narrativeRefinement, memoryConsolidation, memoryV2Consolidation, memoryV3SelectL2, recall, conversationStarters, identityIntro, emptyStateGreeting) default to balanced; everything else (classifiers, summarization, titles, copy generation, memory extraction/retrieval/sweeps, heartbeat, home-screen content, etc.) defaults to cost-optimized. Nothing "falls back" to an expensive model.
• Do not write a full llm.callSites blob that mirrors the shipped defaults. That freezes today's defaults into user config and silently opts the user out of future default improvements (and of tuning shipped alongside them, like cache and context-window settings). Pin only deliberate deviations.

Step 1 — Measure current spend

# Monthly totals
assistant usage totals --range month

# Break down by conversation (what the user actually did — use this for presentation)
assistant usage breakdown --group-by conversation --range month