Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →jeremylongshore/langchain-cost-tuning
Control LangChain 1.0 AI spend with accurate streaming token accounting, model tiering, provider-specific cache hit tuning, per-tenant budgets, and retry dedup.
Passed 18 of 22 test cases with the skill, and 13 of 22 without it.
and 6 more in the full report
6 fixed12 passed either way3 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
Our LangChain streaming dashboard lags behind real execution and double-counts API usage whenever network retries kick in. Can you provide a Python token accounting module that safely extracts unified usage metrics—including cached and reasoning tokens—across different model providers, alongside a streaming event handler that deduplicates retry attempts using a consistent request identifier?
| Checked | Without skill | With skill |
|---|---|---|
| The implementation extracts token counts directly from AIMessage.usage_metadata rather than response_metadata or provider-specific token usage dictionaries. | Missed | Passed |
| The streaming handler uses astream_events with version set to 'v2' to process stream tokens. | Missed | Passed |
| The streaming handler filters on the 'on_chat_model_stream' event type rather than callbacks on 'on_llm_end'. | Missed | Passed |
| The meter deduplicates emissions by tracking a request_id or run_id key to avoid double-counting retries. | Passed | Passed |
The instructions your model reads when the skill loads.
An engineer shipped a new research agent Tuesday. By Friday the Anthropic
bill had grown 6x while traffic grew 1.4x. The cost dashboard — wired to
on_llm_end — showed spend up maybe 2x. Reconciling against the provider
console on Monday surfaced two compounding bugs: (1) the agent's ChatOpenAI
fallback kept the default max_retries=6, so each logical call billed as up
to 7 requests (P30); (2) retry middleware was registered below token
accounting, so every retry fired on_llm_end twice — the aggregator summed
both emissions while LangSmith deduped them by generation ID, undercounting
the dashboard by ~50% against actual billed rate (P25).
The fix took an afternoon: cap retries at 2, tag retries with a stable
request_id, and migrate token accounting to AIMessage.usage_metadata read
from astream_events(version="v2"). Finding the bug took a week. This skill
is that week compressed into a runbook.
Cost tuning for a LangChain 1.0 production app has five levers, each with a sharp failure mode:
on_llm_end lags streams by 5-30s (P01); retries double-count (P25); Anthropic cache savings aggregate per-call, never per-session (P04).max_retries=6 default on ChatOpenAI (P30); Anthropic 50 RPM tier throttles cached and uncached calls against the same budget (P31).create_react_agent defaults to recursion_limit=25; vague prompts burn a session's budget before GraphRecursionError surfaces (P10).InMemoryCache ignores bound tools in the cache key and returns wrong answers (P61); RedisSemanticCache ships with a 0.95 threshold that hits <5% of the time (P62).claude-opus-4-5 on intent classification is 30-60x more expensive than claude-haiku-4-5 for a task the cheaper model solves at equal quality.Pin: langchain-core 1.0.x, langchain-anthropic 1.0.x, langchain-openai 1.0.x.
Pain-catalog anchors: P01, P04, P10, P23, P25, P30, P31, P61, P62.
langchain-core >= 1.0, < 2.0pip install langchain-anthropic langchain-openairedis-py >= 5.0 for budget middleware (optional; in-process dict works for dev)usage_metadata
against billed spend — you will need this to verify any instrumentation fixtest cases passed, vs 13 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.