Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Reproducibility audit — measured AI agent skill | DecimalAI

docxology/template-reproducibility-audit

SkillScore77Maintained

Deterministic reproducibility audit — fixed seeds, regenerate-from-clean, double-run diff before Zenodo/arXiv/release. USE WHEN outputs drift between runs, "worked on my machine", need regenerate-from-clean proof, or pre-release reproducibility check — even without naming docs/prompts.

by docxology

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

Research pack

decimalai skills pull docxology-template-reproducibility-audit

Get this skill — curl · raw file

curl -fsSL https://app.decimal.ai/s/docxology-template-reproducibility-audit/SKILL.md

Raw file: /s/docxology-template-reproducibility-audit/SKILL.md ↗ · machine-readable summary at /s/docxology-template-reproducibility-audit.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 6d ago

—-25% tokens0% turns77% of whole cases pass with skill (17/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-21	✗→✓	▲ Improved	-7%	0%
case-04	✗→✓	▲ Improved	-27%	0%
case-05	✗→✓	▲ Improved	-38%	0%
case-08	✗→✓	▲ Improved	13%	0%
case-09	✗→✓	▲ Improved	-14%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/1/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

docxology / template

docs/prompts/reproducibility-audit/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/1/2026Version v1

Skill contents

SKILL.md

Reproducibility audit

Complements manuscript-claim-verification (claim truth) by focusing on stability of artifacts.

Natural invoke

"Prove template_code_project is reproducible before Zenodo"
"Numbers in the PDF don't match after a clean rebuild"
"Double-run the pipeline and diff outputs"

Inputs to confirm

Project — from docs/_generated/active_projects.md.

Workflow

Determinism — fixed RNG seeds, MPLBACKEND=Agg; no wall-clock/hostname/path leaks. List nondeterministic sources.

Regenerate from clean — wipe working outputs, run core pipeline, regenerate manuscript variables. Capture exit status per stage.

Diff — compare regenerated output/<name>/ and manuscript variables vs prose assertions. Hand-typed numbers not from generated variables are findings even if equal.

Double-run stability — regenerate twice with clean tree between; any run-1 vs run-2 diff is hard failure.

Fix — seed injection, variable-ize hard-typed numbers, remove timestamp leakage. Update projects/<n>/AGENTS.md and README.md with regeneration command. Never hand-edit output/.

Deliverables

Drift table: artifact | committed | regenerated | action.
Commands + raw output; no invented coverage numbers.

Verification commands

bash
uv sync
uv run python scripts/runner/execute_pipeline.py --project <project> --core-only
uv run python projects/<project>/scripts/z_generate_manuscript_variables.py
git stash --include-untracked -- output/<project> 2>/dev/null || true
uv run python scripts/runner/execute_pipeline.py --project <project> --core-only
git status --porcelain output/<project>
uv run python scripts/pipeline/stage_01_test.py --project <project>
uv run python -m infrastructure.validation.cli prerender projects/<project>/manuscript --repo-root .
uv run python -m infrastructure.validation.cli pdf output/<project>/pdf/
uv run python -m infrastructure.validation.cli integrity output/<project>/

When NOT to use

Stage fails during regeneration → pipeline-debugging
Per-sentence claim audit → manuscript-claim-verification

Want to see if docxology/template-reproducibility-audit would activate on your phrasing? Try it before you install — no LLM call, no signup.

Other measured skills in the registry, with their headline benchmark lift.