Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Test the docs · supabase | DecimalAI

testingby supabase

Test the docs

supabase/test-the-docs

Source supabase/supabase ↗License Apache-2.0

Execute runnable docs snippets and examples inside a disposable Docker Compose sandbox (runner container + local Supabase stack via `supabase start`). Use after Draft or during Self-review when asked to test the docs, fact-check CLI/SQL/code samples, or produce a verification report for a docs PR. Complements review-the-docs lint/build checks; does not replace them.

SkillScore77Maintained

Automatic

Doc Ops pack

Last verified 10/6/2026

What it changes

Passed 14 of 18 test cases with the skill, and 5 of 18 without it.

Without skill27%

With skill77%

Checks it passed that the model alone missed

Requires one end-to-end path for Tier A.
Refuses sandbox execution when the host user is root.
Executes ./sandbox/run.sh down for container cleanup.

and 14 more in the full report

11 fixed6 passed either way5 still failing0 broke

Pass-rate lift: +50 pp
Tokens: +18%
Agent turns: 0%
Response time: -49%

Model gemini-3.6-flashMeasured 10/6/2026v117 of 22 cases passed with skill · 18 comparable

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/supabase-test-the-docs/SKILL.md https://app.decimal.ai/s/supabase-test-the-docs/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add supabase-test-the-docs, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/supabase-test-the-docs/ and pins it in .decimal/skills.lock

decimalai skills add supabase-test-the-docs

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/supabase-test-the-docs/SKILL.md https://app.decimal.ai/s/supabase-test-the-docs/SKILL.md

Raw file: /s/supabase-test-the-docs/SKILL.md ↗ · machine-readable summary at /s/supabase-test-the-docs.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 11

Prompt

We are writing an automated wrapper around the docs sandbox test runner. What command must always execute upon completion to tear down container resources?

What was checked, without and with the skill
Checked	Without skill	With skill
Executes ./sandbox/run.sh down for container cleanup.	Missed	Passed

case-10 · gemini-3.6-flash · 10/6/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md86 linessingle file

Test the docs

Runs procedural docs content inside disposable containers, not on the host shell and not against production. Produces a verification report for the PR body / self-review note.

For lint, markdown rebuilds, example-app triage, and PR review, use review-the-docs. For Frame/Shape and cross-repo product lookup, use pm-the-docs.

When to invoke

After Draft, before or during Self-review (checklist Stage 4).
Standalone: "test the docs", "fact-check these snippets", "run the examples".
Content or tutorial PRs that add or change procedural fenced blocks.

Not for: generated reference pages, docs-app architecture questions, or hosted/production projects.

Core rules

Never run against production. Local stack or temp dir only.
Never run MDX fences on the host shell. Use the Compose sandbox — see reference/sandbox-setup.md and sandbox/run.sh.
Proportional: Tier A (one end-to-end path) is required; Tier B spot-checks new/changed procedural blocks, not every fence on every page.
Product bugs found while testing get linked or filed separately; fix docs only when the docs are wrong.

Phases

1. Scope

From explicit MDX paths, or:

git diff --name-only master...HEAD -- 'apps/docs/content/**'

Skip generated reference output under features/docs/generated/.

2. Extract

List runnable artifacts from changed MDX:

Fenced blocks: bash, sh, sql, javascript, typescript, tsx, jsx
$CodeSample paths → treat as example-app (build under examples/)
Skip: mermaid, incomplete illustrative fragments, partial-only includes

Track record

Benchmark

14 / 18

test cases passed, vs 5 / 18 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v5 — 9/8/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

supabase / supabase

.agents/skills/test-the-docs/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 9/8/2026Version v1

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Test the docs

When to invoke

Core rules

Phases

1. Scope

2. Extract

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills