Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Register custom Pi Agent model slugs so saved OpenRouter variants resolve correctly.
.claude/skills/sickn33-pi-custom-model/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-02 | ✗→✓ | ▲ Improved | 18% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 17% | 0% |
| case-04 | ✗→✓ | ▲ Improved | 70% | 0% |
| case-07 | ✗→✓ | ▲ Improved | 23% | 0% |
| case-08 | ✗→✓ | ▲ Improved | 3% | 0% |
Pi's saved default only loads if the exact provider/id exists in its model registry. Pi ships a static bundled list per provider — so OpenRouter routing-shortcut variants (:nitro = sort by throughput, :floor = cheapest, :exacto = quality tool-use) and any brand-new slug are NOT in it. When the default doesn't resolve, Pi silently falls through to its built-in per-provider default (for openrouter that's moonshotai/kimi-k2.6) — looking like Pi "reset" your model. Fix = register the slug as a custom model so find(provider, id) matches.
~/.pi/agent/settings.json — defaultProvider, defaultModel, defaultThinkingLevel~/.pi/agent/models.json — custom models, keyed by provider~/.pi/agent/auth.json — provider credentials (check the provider key exists)auth.json (or an env var like OPENROUTER_API_KEY). No auth → the model is registered but unavailable → still falls back.models.json under providers.<provider>.models. For a built-in provider (openrouter, anthropic, etc.) you only supply metadata — api, baseUrl, and auth are inherited from the bundled defaults. Example:json { "providers": { "openrouter": { "models": [ { "id": "z-ai/glm-5.2:nitro", "name": "Z.ai: GLM 5.2 (nitro)", "reasoning": true, "thinkingLevelMap": { "xhigh": "xhigh" }, "input": ["text"], "cost": { "input": 0.95, "output": 3, "cacheRead": 0.18, "cacheWrite": 0 }, "contextWindow": 1048576, "maxTokens": 32768, "compat": { "supportsDeveloperRole": false, "thinkingFormat": "openrouter" } } ] } } } Copy cost/contextWindow/compat from the base model (the variant shares them) — find the bundled entry in <pi-pkg>/node_modules/@earendil-works/pi-ai/dist/providers/<provider>.models.js. Don't hardcode generic 128k/16k if the real model is bigger.
settings.json: defaultProvider + defaultModel = the exact id. Leave defaultThinkingLevel as the user has it.pi --list-models | grep <id> shows it, and JSON parses. Optionally smoke-test: pi --provider <p> --model "<id>" "which model are you?".find() is exact provider+id — no fuzzy/colon-stripping for the saved default path. The slug in settings.json and models.json must be byte-identical.settings.json alone. Setting defaultModel to an unregistered slug does nothing — models.json is the actual fix.enabledModels (optional) pins the model picker so Ctrl+P cycling can't drift back: "enabledModels": ["<provider>/<id>:<thinking>"]..pi/settings.json overrides global. If a default reverts only inside one project, check that file first.User request:
> Register custom Pi Agent model slugs so saved OpenRouter variants resolve correctly.
davidondrej/skills; verify local paths, tools, credentials, and agent features before acting.| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 11,441 | 6,964 | -39% | 1 | 1 | 0% | 2,153 | 2,449 | +14% | 0 | 0 | — |
case-02 | fail→pass | 11,975 | 8,742 | -27% | 1 | 1 | 0% | 2,277 | 2,679 | +18% | 0 | 0 | — |
case-03 | fail→pass | 12,834 | 7,701 | -40% | 1 | 1 | 0% | 2,393 | 2,799 | +17% | 0 | 0 | — |
case-04 | fail→pass | 9,510 | 9,213 | -3% | 1 | 1 | 0% | 1,716 | 2,917 | +70% | 0 | 0 | — |
case-05 | fail→fail | 14,159 | 7,197 | -49% | 1 | 1 | 0% | 2,360 | 2,311 | -2% | 0 | 0 | — |
case-06 | fail→fail | 13,048 | 9,152 | -30% | 1 | 1 | 0% | 2,358 | 2,745 | +16% | 0 | 0 | — |
case-07 | fail→pass | 10,565 | 5,661 | -46% | 1 | 1 | 0% | 1,801 | 2,210 | +23% | 0 | 0 | — |
case-08 | fail→pass | 10,301 | 5,011 | -51% | 1 | 1 | 0% | 2,013 | 2,083 | +3% | 0 | 0 | — |
case-09 | fail→pass | 10,648 | 8,479 | -20% | 1 | 1 | 0% | 2,039 | 2,835 | +39% | 0 | 0 | — |
case-10 | fail→pass | 11,939 | 7,620 | -36% | 1 | 1 | 0% | 2,003 | 2,424 | +21% | 0 | 0 | — |
case-11 | fail→pass | 10,717 | 6,601 | -38% | 1 | 1 | 0% | 2,004 | 2,389 | +19% | 0 | 0 | — |
case-12 | pass→pass | 8,932 | 4,568 | -49% | 1 | 1 | 0% | 1,632 | 1,842 | +13% | 0 | 0 | — |
case-13 | fail→pass | 8,131 | 4,350 | -47% | 1 | 1 | 0% | 1,372 | 1,914 | +40% | 0 | 0 | — |
case-14 | fail→pass | 9,665 | 4,376 | -55% | 1 | 1 | 0% | 1,655 | 1,870 | +13% | 0 | 0 | — |
case-15 | pass→pass | 8,661 | 3,779 | -56% | 1 | 1 | 0% | 1,482 | 1,682 | +13% | 0 | 0 | — |
case-16 | pass→pass | 14,004 | 6,525 | -53% | 1 | 1 | 0% | 2,589 | 2,513 | -3% | 0 | 0 | — |
case-17 | fail→pass | 16,439 | 9,122 | -45% | 1 | 1 | 0% | 2,859 | 2,747 | -4% | 0 | 0 | — |
case-18 | pass→pass | 9,499 | 2,234 | -76% | 1 | 1 | 0% | 1,496 | 1,388 | -7% | 0 | 0 | — |
case-19 | pass→pass | 12,267 | 7,279 | -41% | 1 | 1 | 0% | 2,154 | 2,452 | +14% | 0 | 0 | — |
case-20 | pass→pass | 8,664 | 3,644 | -58% | 1 | 1 | 0% | 1,834 | 1,601 | -13% | 0 | 0 | — |
case-21 | pass→pass | 9,339 | 3,608 | -61% | 1 | 1 | 0% | 1,701 | 1,605 | -6% | 0 | 0 | — |
case-22 | pass→pass | 10,238 | 4,033 | -61% | 1 | 1 | 0% | 1,748 | 1,699 | -3% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +50 percentage points is the difference between those two pass rates over the 22 comparable cases.
The publisher has shipped newer versions since this run, so these numbers describe v1, not the version currently listed.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.