Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Explain plugin, skill, command, agent, and hook mechanics used here. Use when authoring or debugging plugins. Do not use for ops; use night-market-operations.
.claude/skills/athola-claude-code-plugin-reference/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-02 | ✗→✓ | ▲ Improved | 298% | 0% |
| case-01 | ✗→✓ | ▲ Improved | 245% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 257% | 0% |
| case-04 | ✗→✓ | ▲ Improved | 231% | 0% |
| case-06 | ✗→✓ | ▲ Improved | 1206% | 0% |
This is the domain pack for how Claude Code plugin machinery works in this repository. It covers manifests, skills, commands, agents, hooks, and the marketplace registry, with the local conventions layered on top. Every contract below was checked against a shipped plugin (mostly plugins/imbue, plugins/abstract, and plugins/herald) on 2026-07-02 at repo v1.9.15. When another night-market skill says "see the plugin reference for what a hook payload is," this is the file it means.
Each term is defined once here and used without redefinition below.
| Term | Meaning in this repo | |------|----------------------| | Plugin | A directory under plugins/ shipping skills, commands, agents, and hooks, with its own tests, Makefile, and pyproject.toml. Independently deployable (ADR-0001). | | Skill | A directory skills/<name>/ containing SKILL.md. Loaded by the model on demand, invoked as Skill(plugin:name). | | Module | A file under a skill's modules/ subdirectory. Loaded only when the hub SKILL.md routes to it (progressive loading). | | Command | A markdown file under commands/. Surfaces as a slash command the user types. Repo docs refer to the namespaced form, e.g. /sanctum:sync-capabilities. | | Agent | A markdown file under agents/ defining a dispatchable subagent: frontmatter plus a system-prompt body. | | Hook | An executable script Claude Code runs on a lifecycle event, registered in the plugin's hooks/hooks.json. | | Hook event | The lifecycle point a hook fires on: PreToolUse, PostToolUse, Stop, SessionStart, UserPromptSubmit, and others (full table below). | | Manifest | .claude-plugin/plugin.json: the file Claude Code reads to register a plugin's components. | | Marketplace | The root .claude-plugin/marketplace.json registry listing all plugins, installed via /plugin marketplace add. | | Plugin cache | The directory Claude Code copies installed plugins into and executes them from. It is NOT the repo checkout. | | ${CLAUDE_PLUGIN_ROOT} | Placeholder Claude Code expands to the plugin's cache path. The only safe way to reference plugin files from hooks. | | Host Python | The system Python interpreter that runs hook scripts. Assume 3.9, even though plugin package code targets 3.12. |
Verified tree (imbue, trimmed):
plugins/imbue/
├── .claude-plugin/
│ ├── plugin.json # Claude Code manifest: registers everything
│ └── metadata.json # inter-plugin dependency manifest
├── openpackage.yml # cross-framework export manifest
├── skills/<name>/
│ ├── SKILL.md # hub document with frontmatter
│ └── modules/*.md # spokes, loaded on demand
├── commands/*.md # slash commands
├── agents/*.md # subagents
├── hooks/
│ ├── hooks.json # hook registration (auto-loaded)
│ ├── *.py, *.sh # hook scripts
│ └── shared/ # vendored helpers (hook_io, vow_utils, ...)
├── scripts/ # Python utilities skills shell out to
├── tests/ # pytest suite, run per plugin
├── Makefile # per-plugin test/lint/typecheck targets
├── pyproject.toml # package config, per-plugin coverage threshold
└── README.md| Field | What it does | Local rule | |-------|--------------|------------| | name | Plugin identity and namespace prefix for skills/commands | Matches directory name | | version | Plugin version | Lockstep with marketplace version (1.9.15). Never hand-edit, use the bumper (below) | | description | Marketplace listing text | Two-part style per ADR-0003 | | skills | Array of "./skills/<dir>" paths. Each dir must contain a SKILL.md. Registration makes Skill(plugin:name) resolvable | Keep in sync with disk via /sanctum:update-plugins | | commands | Array of "./commands/<file>.md" paths, each becoming a slash command | Same sync rule | | agents | Array of "./agents/<file>.md" paths, each a dispatchable subagent | Same sync rule | | hooks | Paths to ADDITIONAL hook files only. ./hooks/hooks.json is auto-loaded and must never appear here | Every plugin in this repo keeps "hooks": [] (verified across all 23 manifests). See the hooks section for why | | keywords | Marketplace search and discovery terms | Free-form | | dependencies / optional_dependencies | Names of other night-market plugins | Informational plus runtime detection only. ADR-0001 forbids import coupling: plugins detect each other via the filesystem and degrade gracefully | | author, license | Attribution | MIT throughout |
Second manifest in .claude-plugin/. Fields verified on imbue: name, version (lockstep), main, a skills subset, dependencies, provides (infrastructure/patterns/tools), and a claude block (skill_prefix, auto_load, categories). One trap: dependency ranges here use their own semver namespace. imbue declares "abstract": ">=2.0.0" while every plugin ships 1.9.15. The range is not compared against the marketplace version, so do not "fix" it to 1.9.15 during a bump. The update_versions.py bumper knows which fields to touch.
Cross-framework manifest consumed by the export pipeline (make cross-framework, scripts/clawhub_export.py). Lists the skills exported outside Claude Code and dependency sources in gh@athola/claude-night-market form. Its version is also lockstep.
Fields verified on plugins/imbue/skills/proof-of-work/SKILL.md. Only name and description are consumed by Claude Code itself. The rest are house conventions read by repo tooling.
| Field | Purpose | |-------|---------| | name | Skill identity. Skill(plugin:name) resolves against it | | description | Loading trigger. Hard cap 160 chars, enforced by plugins/abstract/scripts/validate_budget.py (DESCRIPTION_MAX = 160, ecosystem budget 90,000 chars per ADR-0004). House template: verb phrase, then "Use when trigger].", then "Do not use when negative]; use sibling] instead." | | role | One of entrypoint, library, hook-target (taxonomy below) | | modules | List of modules/*.md spokes this hub can load | | dependencies, tags, category, usage_patterns | Discovery metadata | | estimated_tokens, complexity, model_hint, alwaysApply, tools | Advisory metadata for tooling and audits |
Role taxonomy (from docs/skill-integration-guide.md):
| Role | Inbound refs | User-invoked | Example | |------|--------------|--------------|---------| | entrypoint | low (0-3) | yes | sanctum:do-issue | | library | high (4+) | rarely | imbue:proof-of-work | | hook-target | varies | no | imbue:vow-enforcement |
A skill's SKILL.md is the hub: it carries the always-loaded core and a routing table into modules/*.md spokes. The model loads a spoke only when the hub tells it to. This keeps the initial token cost near the estimated_tokens value instead of the whole directory. The pattern is specified in Skill(leyline:progressive-loading).
SKILL.md needs a ## Exit Criteria section withconcrete, observable, falsifiable checkboxes (.claude/rules/skill-exit-criteria.md). A pre-commit ratchet, scripts/check_skill_exit_criteria_drift.py, fails when the count of missing sections rises above scripts/skill_exit_criteria_baseline.json.
Skill(plugin:name) references in prose are scanned byplugins/abstract/scripts/skill_graph.py. A second ratchet, scripts/check_skill_graph_drift.py, blocks commits that add dangling references (the bugs category) above scripts/skill_graph_baseline.json. When your change lowers the count, lower the baseline too so the ratchet tightens.
structural validation test per new skill (constitution names the pattern test_skill_<name>.py; in practice imbue keeps them at tests/unit/skills/test_<name>.py, e.g. test_proof_of_work.py).
Skill(imbue:proof-of-work) resolves imbue to the plugin registered in the marketplace, then proof-of-work to the directory listed in that plugin's plugin.json skills array. If the directory or the manifest entry is missing, the reference is dangling and the skill-graph ratchet will flag it.
Frontmatter verified on plugins/imbue/commands/justify.md and plugins/sanctum/commands/fixit.md:
yaml--- name: fixit description: Fix broken functionality from pasted error output... usage: /fixit <pasted-error-or-description> [--file <path>] ... extends: "imbue:proof-of-work" ---
name and description are required. usage documents the argumentshape. extends (optional) names the skill the command routes into.
$ARGUMENTS in the body is replaced with the raw argument string theuser typed after the command (see fixit: "The input is $ARGUMENTS (a paste) or the contents of --file <path>").
When to write a command versus a skill:
| Write a | When | |---------|------| | Command | The user should type it deliberately with arguments (an entrypoint with a CLI-like contract) | | Skill | The model should load it when a trigger matches (knowledge, procedure, or a library other skills compose) | | Both | Common pattern here: a thin command that extends or routes into a skill (/imbue:justify wraps imbue:justify) |
The Exit Criteria rule does not apply to command files.
Frontmatter verified on plugins/imbue/agents/review-analyst.md:
yaml--- name: review-analyst description: Autonomous agent for conducting structured reviews with evidence gathering. Use when detailed code reviews requiring evidence trails... Do not use when quick code check without formal review... model: opus tools: - Read - Glob - Grep - Bash skills: imbue:review-core, imbue:proof-of-work, ... ---
description drives dispatch: Claude decides whether to hand a taskto the agent by matching it, so it must carry "Use when" and "Do not use when" clauses just like a skill description.
tools is the allowlist the subagent gets. model pins the model tier.skills lists skills the agent should load. The body below thefrontmatter is the agent's system prompt.
The hardest-won knowledge in the repo. Read this whole section before writing or editing any hook.
Survey of every plugins/*/hooks/hooks.json (2026-07-02):
| Event | Registered by | |-------|---------------| | PreToolUse | abstract, conserve, gauntlet, imbue, leyline, memory-palace, pensive, sanctum | | PostToolUse | abstract, cartograph, conserve, gauntlet, leyline, memory-palace, sanctum | | UserPromptSubmit | abstract, egregore, imbue, memory-palace, sanctum | | SessionStart | conserve, egregore, imbue, leyline, memory-palace, oracle, sanctum, tome | | Stop | abstract, egregore, herald, memory-palace, oracle, sanctum | | PreCompact | conserve, tome | | Setup, PermissionRequest, PermissionDenied | conserve | | ConfigChange | sanctum | | Notification | conserve |
| Event | Shipped | Fires when | |-------|---------|------------| | DirectoryAdded | 2.1.220 | /add-dir or an SDK register_repo_root registers a working directory mid-session |
Notification (2.1.218) fires when a background agent needs input or completes. conserve records each event to .claude/logs/. Anthropic publishes the registration shape but not the stdin fields, so the hook writes the payload through verbatim and records its keys. Read the log before writing code against a field name.
SessionStart gained a fifth source value in 2.1.212: a session opened as a fork reports "fork". A matcher that enumerates the other four silently stops firing for forks, which is what tests/unit/test_session_start_sources.py now guards. conserve covers every source. egregore matches only startup and resume: fork and compact continue a conversation that already carries its banner, and clear was excluded before fork shipped, so this change leaves that choice as it found it.
Hook if: conditions changed in 2.1.214. A single-segment dir/** pattern now matches only <cwd>/dir, so a hook meant to fire at any depth needs **/dir/**. Note that deny and ask permission rules kept their any-depth behavior, so the two syntaxes no longer agree and copying a pattern from one to the other silently changes its scope.
Claude Code auto-loads hooks/hooks.json from each installed plugin. Listing ./hooks/hooks.json in the plugin.json hooks array causes a "Duplicate hooks file" error at session start. The pre-commit guard scripts/check_plugin_hooks.py rejects any manifest that does it. The hooks array exists only for additional hook files beyond the auto-loaded default, and every plugin here keeps it [].
Shape (verified, herald):
json{ "hooks": { "Stop": [ { "hooks": [ { "type": "command", "command": "${CLAUDE_PLUGIN_ROOT}/hooks/double_shot_latte.py", "timeout": 10 } ] } ] } }
PreToolUse/PostToolUse entries add a matcher regex over tool names (imbue uses "Write|Edit|MultiEdit" and "Bash"). timeout is the per-hook budget in seconds. Always reference the script through ${CLAUDE_PLUGIN_ROOT}.
Claude Code delivers the payload as one JSON object on stdin with fields such as tool_name, tool_input, tool_response, session_id, and hook_event_name. It does NOT set CLAUDE_TOOL_NAME, CLAUDE_TOOL_INPUT, or CLAUDE_TOOL_OUTPUT environment variables. Hooks that read those env vars silently no-op on every real invocation. That exact bug left the skill-execution logger dead for months with no error anywhere (fixed in 1.9.14). Canonical read pattern:
pythonimport json import sys def main() -> None: try: payload = json.loads(sys.stdin.read()) except (json.JSONDecodeError, OSError): sys.exit(0) # fail open: a broken hook must not wedge the session tool_name = payload.get("tool_name", "") tool_input = payload.get("tool_input", {}) command = tool_input.get("command", "")
The shared reader plugins/abstract/hooks/shared/hook_io.py (read_hook_payload()) implements stdin-first with a legacy env fallback for the synthetic test harness, and warns on stderr when stdin JSON is malformed. Because ADR-0001 forbids cross-plugin imports, leyline (hooks/noqa_guard.py) and sanctum (hooks/deferred_item_watcher.py) carry parallel copies that must change together (noted in the hook_io docstring). Hooks import their own plugin's shared/ sibling by inserting the script directory on sys.path first:
pythonfrom pathlib import Path sys.path.insert(0, str(Path(__file__).parent)) from shared.vow_utils import is_git_commit, shadow_mode_active
Repo convention is exit 0 always, with the verdict carried as JSON on stdout. Comments in plugins/imbue/hooks/tdd_bdd_gate.py record that exit code 2 has blocked the tool call since Claude Code v2.1.90, so a stray nonzero exit is itself a decision. Fail open on parse errors.
| Event | Verdict shape (verified in repo hooks) | |-------|-----------------------------------------| | PreToolUse | {"hookSpecificOutput": {"hookEventName": "PreToolUse", "permissionDecision": "allow" \| "deny" \| "ask", "permissionDecisionReason": "..."}}. Those three are the only values the harness recognizes (see plugin-dev:hook-development and the upstream hook docs). Correct exemplar: plugins/leyline/hooks/noqa_guard.py emits "deny". Print nothing (exit 0) to allow | | Stop | {"decision": "block", "reason": "..."} forces the session to keep working. {"decision": "approve", ...} allows the stop (herald double_shot_latte.py) | | UserPromptSubmit | {"hookSpecificOutput": {..., "additionalContext": "..."}} injects context (abstract pre_skill_execution.py, aggregate_learnings_daily.py) |
Human-facing diagnostics go to stderr, never stdout (stdout must stay parseable JSON).
Repo-local anomaly, do not copy: imbue's shadow-mode vow hooks (vow_no_ai_attribution.py) emit "warn" in shadow mode and "block" when VOW_SHADOW_MODE=0. Neither value is in the harness contract (allow/deny/ask), so the hook most likely fails open silently even with shadow mode off: the SB9 failure class from night-market-failure-archaeology. vow_bounded_reads.py in the same plugin correctly emits "deny". File an issue rather than imitating the "warn"/"block" output. (Anomaly flagged 2026-07-03.)
Any subprocess a hook spawns must finish inside the timeout registered in hooks.json, with headroom. herald once shipped an LLM call whose timeout exceeded its registered Stop-hook budget, so the harness killed the hook before any verdict was emitted (full record: night-market-failure-archaeology SB7). Copy the fix's pattern: assert subprocess_timeout < registered_budget in a guard test, not a comment.
Plugin package code targets Python 3.12, but hook scripts run under the system interpreter, assumed 3.9. python39-compat.yml enforces two gates with uneven coverage: ruff UP007 under --target-version py39 (bare unions) runs on 12 plugins' hooks/**, while the hook test subtree in a real 3.9 venv runs on only 7 plugins (abstract, conserve, egregore, imbue, leyline, memory-palace, sanctum). herald ships a Stop hook yet is covered by neither gate.
| Banned in hook import chains | Use instead | |------------------------------|-------------| | datetime.UTC (a 3.11+ alias that broke hooks 3+ times, and ruff UP017 kept auto-reverting the fix, so root ruff config carries extend-ignore UP017) | from datetime import timezone then datetime.now(timezone.utc) | | Bare X \| Y annotations without from __future__ import annotations | Put the future import first in every hook file | | Unguarded third-party imports (yaml, anthropic) anywhere a hook's import chain reaches | Guard with try/except ImportError or import lazily inside the function. An eager import in gauntlet/__init__.py once broke every git commit with a PreToolUse ModuleNotFoundError |
The durable defense against datetime.UTC regressions is an AST-scanning test, not a lint rule: see plugins/leyline/tests/test_python39_compat.py, which walks the source tree and fails on any reintroduction. When a linter fights your fix, add an AST invariant test.
Installed plugins run from the Claude Code plugin cache, not the repo. Consequences:
hooks.json. Use${CLAUDE_PLUGIN_ROOT} (config) or Path(__file__) (Python).
Shared helpers are vendored per plugin: each hook-bearing plugin keeps a byte-identical copy of scripts/shared/json_utils.sh under its own hooks/shared/, and make check-json-utils fails on drift.
Hook behavior is tested by piping a synthetic payload:
bashecho '{"tool_name": "Bash", "tool_input": {"command": "git commit -m x"}}' \ | python3 plugins/imbue/hooks/vow_no_ai_attribution.py
Exit code and stdout JSON are the assertions. Hook test subtrees (for example plugins/imbue/tests/unit/hooks/) run under 3.9 in CI, so keep them stdlib-plus-pytest only.
.claude-plugin/marketplace.json. Top-level name,version, description, owner, and a plugins array of {name, source: "./plugins/<dir>", description, version, keywords}.
/plugin marketplace add athola/claude-night-market(README quick start), then /plugin install <name>@claude-night-market. The install form is documented in the book's getting-started pages; re-verify there if it fails.
to every plugin's plugin.json, metadata.json, openpackage.yml, pyproject.toml, and package __init__.py. Bump with:
bashuv run python plugins/sanctum/scripts/update_versions.py 1.9.16 --dry-run uv run python plugins/sanctum/scripts/update_versions.py 1.9.16
/sanctum:update-plugins [plugin] [--fix] auditsplugin.json arrays against disk contents. make docs-sync-check (and the capabilities-sync.yml CI job) verifies book/src/reference/capabilities-*.md matches registrations. Fix drift with /sanctum:sync-capabilities --fix, never by hand-editing the generated capabilities files.
book/src/ (mdBook, published by deploy-book.yml).book/src/reference/capabilities-*.md are generated by the sync tooling. Getting-started and plugin pages are hand-written.
docs/plugin-development-guide.md,docs/skill-description-guide.md, docs/skill-integration-guide.md. Note: the dev guide's make create-plugin NAME=... target does not exist in the root Makefile as of 2026-07-02 (stale doc). Scaffold with Skill(abstract:create-skill) and copy an existing plugin's layout.
README badge). Consult upstream when the question is about harness behavior this repo only records as code comments: full event payload schemas, new hook events, or exit-code semantics changing across Claude Code versions.
| You need | Use instead | |----------|-------------| | Run tests, lint, typecheck, release, publish | night-market-operations | | Classify or gate a change, review rules | night-market-change-control | | Recreate the dev environment (uv, tool pins) | night-market-build-and-env | | Triage a failing hook or CI job by symptom | night-market-debugging-playbook | | History of why a hook or pattern exists | night-market-failure-archaeology | | Design invariants across plugins (isolation, lockstep rationale) | night-market-architecture-contract | | Config axes and env vars (VOW_SHADOW_MODE and friends) | night-market-config-catalog | | Evidence bar and test-discipline policy | night-market-validation-and-qa |
hooks/hooks.json only, and python3 scripts/check_plugin_hooks.py exits 0 afterward.
CLAUDE_TOOL_* env vars) and exits 0 when fed malformed input: echo not-json | python3 <hook>.py; echo $? prints 0.
be strictly less than the timeout registered in hooks.json.
from __future__ import annotations,uses timezone.utc rather than datetime.UTC, imports no unguarded third-party package, and the python39-compat.yml gates pass.
plugin.json skillsarray, its description is at most 160 characters, it has an ## Exit Criteria section, and both ratchet scripts (check_skill_graph_drift.py, check_skill_exit_criteria_drift.py) pass.
plugin.json hooks arrays are emptyin this repo and what error appears if they are not.
Compiled 2026-07-02 against repo v1.9.15 (branch discussions-fix-1.9.14). PreToolUse verdict contract corrected to allow|deny|ask and the imbue vow-hook warn/block anomaly flagged on 2026-07-03. Volatile facts and how to re-verify them:
bash# Hook events per plugin for f in plugins/*/hooks/hooks.json; do echo "$f" python3 -c "import json,sys; print(list(json.load(open(sys.argv[1]))['hooks']))" "$f" done # All plugin.json hooks arrays empty (exit 0 = clean) python3 scripts/check_plugin_hooks.py rg '"hooks": \[' plugins/*/.claude-plugin/plugin.json # 160-char description cap still in force rg -n 'DESCRIPTION_MAX' plugins/abstract/scripts/validate_budget.py # Payload-on-stdin contract (read the module docstring) head -15 plugins/abstract/hooks/shared/hook_io.py # Exit-2 semantics comment rg -n 'exit 2 blocks' plugins/imbue/hooks/tdd_bdd_gate.py # permissionDecision values in repo hooks (deny is correct; the # warn/block hits in vow_no_ai_attribution.py are the known anomaly) rg -n '"permissionDecision"' plugins/*/hooks/*.py # Herald timeout pairing (registered 10s vs internal 8s) rg -n 'LLM_TIMEOUT_SECONDS|timeout' plugins/herald/hooks/ # py39 gates and covered plugins head -50 .github/workflows/python39-compat.yml # Version lockstep fields touched by the bumper rg -n 'def main' -A 12 plugins/sanctum/scripts/update_versions.py # Marketplace entry shape head -60 .claude-plugin/marketplace.json # Stale make create-plugin claim (doc says it exists, Makefile disagrees) rg -n 'create-plugin' Makefile docs/plugin-development-guide.md
Unverified/candidate items are labeled inline: the /plugin install <name>@claude-night-market form (book-documented, not exercised here) and the explanation that metadata.json dependency ranges form a separate semver namespace (observed, not specified anywhere).
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-02 | fail→pass | 12,982 | 10,092 | -22% | 1 | 1 | 0% | 2,246 | 8,945 | +298% | 0 | 0 | — |
case-01 | fail→pass | 15,851 | 17,217 | +9% | 1 | 1 | 0% | 3,016 | 10,408 | +245% | 0 | 0 | — |
case-03 | fail→pass | 16,354 | 11,866 | -27% | 1 | 1 | 0% | 2,580 | 9,209 | +257% | 0 | 0 | — |
case-04 | fail→pass | 14,528 | 5,261 | -64% | 1 | 1 | 0% | 2,410 | 7,983 | +231% | 0 | 0 | — |
case-05 | pass→pass | 11,920 | 6,849 | -43% | 1 | 1 | 0% | 2,000 | 8,320 | +316% | 0 | 0 | — |
case-06 | fail→pass | 37,605 | 6,607 | -82% | 1 | 1 | 0% | 625 | 8,162 | +1206% | 0 | 0 | — |
case-07 | fail→pass | 12,137 | 5,468 | -55% | 1 | 1 | 0% | 2,043 | 8,057 | +294% | 0 | 0 | — |
case-08 | fail→pass | 13,479 | 3,241 | -76% | 1 | 1 | 0% | 2,358 | 7,691 | +226% | 0 | 0 | — |
case-09 | pass→pass | 13,069 | 4,527 | -65% | 1 | 1 | 0% | 2,067 | 7,789 | +277% | 0 | 0 | — |
case-10 | pass→pass | 13,411 | 3,608 | -73% | 1 | 1 | 0% | 1,964 | 7,625 | +288% | 0 | 0 | — |
case-11 | fail→pass | 16,413 | 7,551 | -54% | 1 | 1 | 0% | 2,708 | 8,436 | +212% | 0 | 0 | — |
case-12 | pass→pass | 11,212 | 3,834 | -66% | 1 | 1 | 0% | 1,846 | 7,737 | +319% | 0 | 0 | — |
case-13 | fail→pass | 11,729 | 4,240 | -64% | 1 | 1 | 0% | 1,912 | 7,711 | +303% | 0 | 0 | — |
case-14 | pass→pass | 7,953 | 2,328 | -71% | 1 | 1 | 0% | 1,277 | 7,457 | +484% | 0 | 0 | — |
case-15 | fail→pass | 12,869 | 2,827 | -78% | 1 | 1 | 0% | 1,923 | 7,536 | +292% | 0 | 0 | — |
case-16 | fail→pass | 10,646 | 3,216 | -70% | 1 | 1 | 0% | 1,732 | 7,564 | +337% | 0 | 0 | — |
case-17 | fail→pass | 16,118 | 4,070 | -75% | 1 | 1 | 0% | 2,496 | 7,725 | +209% | 0 | 0 | — |
case-18 | pass→pass | 10,366 | 3,673 | -65% | 1 | 1 | 0% | 1,685 | 7,635 | +353% | 0 | 0 | — |
case-19 | fail→pass | 21,619 | 2,482 | -89% | 1 | 1 | 0% | 3,362 | 7,437 | +121% | 0 | 0 | — |
case-20 | pass→pass | 3,777 | 3,122 | -17% | 1 | 1 | 0% | 604 | 7,584 | +1156% | 0 | 0 | — |
case-21 | pass→pass | 4,646 | 4,894 | +5% | 1 | 1 | 0% | 785 | 7,820 | +896% | 0 | 0 | — |
case-22 | pass→pass | 3,148 | 4,330 | +38% | 1 | 1 | 0% | 511 | 7,775 | +1422% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 21 counted toward the lift figure. The other 1 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +59 percentage points is the difference between those two pass rates over the 21 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.