Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →jpoindexter/model-routing-and-fallback
Reference-grade guide to routing LLM requests across models by cost/latency/quality, escalating hard tasks, and surviving provider failures with multi-provider fallback, circuit breakers, hedging, and honest degraded-mode UX.
Passed 19 of 23 test cases with the skill, and 18 of 23 without it.
2 fixed17 passed either way3 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 2 test cases the skill fixed. Same model, same prompt, with and without the skill.
Design an escalation evaluator for a customer support AI agent. The function checks model responses to see if the query should be re-routed to a higher-tier model or human escalation. What specific output signals should trigger escalation?
| Checked | Without skill | With skill |
|---|---|---|
| Validation schema failures trigger tier escalation. | Passed | Passed |
| Model refusals trigger tier escalation. | Missed | Passed |
The instructions your model reads when the skill loads.
Two problems share this skill. Routing = picking the cheapest model that still meets the quality bar for this request (a value decision). Fallback = surviving when the chosen model is unavailable (a reliability decision). Build both — routing without fallback is a single point of failure; fallback without routing burns money on easy work.
Tiers, not model IDs, are the durable unit: frontier (most capable, slowest, ~10–30× cost), mid (workhorse), small (fast, cheap, weak). Names shift quarterly; the tier logic does not.
The frontier model is the wrong default. ~70–90% of production traffic is easy (classification, extraction, short rewrites, routine Q&A) and a small model handles it at a fraction of the cost and latency. Routing trades three axes:
| Axis | Frontier tier | Mid tier | Small tier |
|---|---|---|---|
| Cost / 1M tok | $$$ (10–60×) | $$ (3–8×) | $ (1×) |
| Latency (p50 TTFT) | high | medium | low |
| Quality ceiling | highest | good | adequate-on-easy |
| Best for | reasoning, ambiguity, long-horizon agents | most tasks | classify, extract, format |
The win is asymmetric: routing 80% of easy traffic to a small model can cut spend 60–80% with near- zero quality delta — if you escalate the hard 20% correctly. The risk is asymmetric too: route a hard task to a weak model and the failure is silent (a confident wrong answer), worse than a catchable error.
Do measure your difficulty distribution before picking a default tier. Don't pick by gut — "use the best model" is a budget decision in disguise.
Map known task types to tiers with a static table. Highest ROI, zero added latency, fully debuggable.
test cases passed, vs 18 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.