Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Pipeline debugging · docxology | DecimalAI

docxology/template-pipeline-debugging

SkillScore82Maintained

Systematic pipeline DAG failure triage for the Research Project Template. USE WHEN ./run.sh or execute_pipeline.py fails, a stage stalls (setup, tests, analysis, render, validate, LLM, copy), pytest/coverage gate fails mid-pipeline, PDF render or validate breaks, Project Analysis finishes too fast with no figures, or user says pipeline debug, stage failed, resume checkpoint, core-only triage — even without naming this skill or docs/prompts.

DevOpsby docxology

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

Doc Ops pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/docxology-template-pipeline-debugging/SKILL.md

decimalai skills pull docxology-template-pipeline-debugging --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/docxology-template-pipeline-debugging/SKILL.md https://app.decimal.ai/s/docxology-template-pipeline-debugging/SKILL.md

Raw file: /s/docxology-template-pipeline-debugging/SKILL.md ↗ · machine-readable summary at /s/docxology-template-pipeline-debugging.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 54d ago

—-19% tokens0% turns82% of whole cases pass with skill (18/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-04	✗→✓	▲ Improved	-12%	0%
case-06	✗→✓	▲ Improved	-22%	0%
case-07	✗→✓	▲ Improved	-12%	0%
case-08	✗→✓	▲ Improved	34%	0%
case-09	✗→✓	▲ Improved	-25%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/1/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

docxology / template

docs/prompts/pipeline-debugging/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/1/2026Version v1

Skill contents

SKILL.md

Pipeline debugging

Natural invoke

"My pipeline failed at PDF render for template_code_project"
"Project Analysis finishes in under a second with no figures — help debug"
"run.sh --pipeline failed; what's the first real error?"
"Resume from checkpoint after fixing project tests"

Inputs to confirm

Project — from docs/_generated/active_projects.md; infer from context if obvious.
Invocation — full vs --core-only, --resume, --skip-infra, multi-project.
Failing stage — if unknown, reproduce first.

Workflow

Reproduce — re-run the failing invocation verbatim; capture full stderr/stdout and exit status. Quote the first real error, not the last line. Name the failing stage (setup, infra tests, project tests, analysis, render, validate, LLM review, LLM translations, copy).

Isolate — run that stage's underlying command directly (pytest target, analysis script, validation CLI, renderer). Use --resume to skip good upstream stages while iterating.

Classify — dependency/uv gap, missing system tool (LaTeX/pandoc-crossref), nondeterministic input, coverage gate, undefined citation/cross-ref, thin-orchestrator violation (logic in scripts/ not src/), or logic bug. When Project Analysis completes in under a second with no figures, treat as import/dependency failure and isolate via scripts/pipeline/stage_02_analysis.py. Trace where bad state enters; fix at ingestion.

Fix minimally — re-run full pipeline green; confirm --resume shows no upstream regression. Update projects/<n>/AGENTS.md / README.md if the failure mode was non-obvious.

Deliverables

Failing stage, first real error (quoted), root cause, minimal patch (file + snippet).
Exact commands + raw exit status proving green.
Do not claim "fixed" without a clean full-pipeline run.

Verification commands

bash
uv sync
uv run python scripts/runner/execute_pipeline.py --project <project>
uv run python scripts/runner/execute_pipeline.py --project <project> --resume
uv run python scripts/runner/execute_pipeline.py --project <project> --core-only
uv run python scripts/pipeline/stage_01_test.py --project <project>
uv run pytest projects/<project>/tests/ --cov=projects/<project>/src --cov-fail-under=90 -q
uv run python -m infrastructure.validation.cli prerender projects/<project>/manuscript --repo-root .
uv run python -m infrastructure.validation.cli pdf output/<project>/pdf/
uv run python -m infrastructure.rendering.latex_package_validator
LOG_LEVEL=0 uv run python scripts/runner/execute_pipeline.py --project <project> --resume

Stage semantics: docs/RUN_GUIDE.md, AGENTS.md.

When NOT to use

Determinism / double-run drift only → reproducibility-audit
Claim truth in prose → manuscript-claim-verification
Validation infrastructure deep dive without a failing stage → validation-quality

Illustrative Want to see what docxology/template-pipeline-debugging would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/4/2026, 00:34 UTC · 22 cases

Measured on 19 comparable cases — below the 22 needed to publish a lift, so this result is shown here but not on the registry card or leaderboard.

Lift

+59%

82% vs 23%

Agent turns

1.0 vs 1.0

Tokens

-19%

1,409.84 vs 1,732.11

fail → pass · 13pass → pass · 5fail → fail · 4

Per-case results (22)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-02 When running the pipeline for `climate_trends`, the data calculation step completes in less than a second without generating any figure outputs. Please investigate what went wrong. Give me a breakdown containing the exact failed stage, the initial error snippet, the underlying cause, the necessary code fix, and proof from verification runs showing successful execution.	fail→fail	4,848	4,817	-1%	1	1	0%	251	1,062	+323%	0	0	—
case-03 My command line build script returned a non-zero exit status while processing project `econ_policy_doc`, generating a massive log output. Please isolate the problem for me: point out which stage failed, quote the true initial error message, diagnose the cause, and output a minimal patch alongside the command outputs proving a successful end-to-end run.	fail→fail	4,053	4,846	+20%	1	1	0%	200	1,156	+478%	0	0	—
case-01 The automated build for my manuscript project `bio_genomics_v2` failed unexpectedly during execution. Can you help debug this? I need you to identify the failing step, quote the first real error line from the output, explain the root cause, and provide a minimal patch file and snippet. Also, provide the exact commands and exit statuses demonstrating that the full run is completely green.	fail→fail	3,877	4,604	+19%	1	1	0%	141	1,177	+735%	0	0	—
case-04 When trying to fix a LaTeX rendering issue in manuscript project `astro_physics`, running the entire build pipeline from start to finish takes 15 minutes each iteration. What command flag pattern should be used to bypass already successful upstream stages while isolating and fixing the renderer?	fail→pass	12,315	5,137	-58%	1	1	0%	1,977	1,746	-12%	0	0	—
case-05 During debugging of project `neuro_img`, I noticed data transformation logic was added directly inside `scripts/runner/execute_pipeline.py` to fix a calculation error. Is this the standard architectural location for stage logic?	pass→pass	10,209	5,618	-45%	1	1	0%	1,806	1,849	+2%	0	0	—
case-06 Project `quantum_sim`'s analysis step returned code 0 in 400ms without writing any figures to disk. The team assumes the numerical algorithms ran fine because no error was thrown. How should this scenario be classified?	fail→pass	11,235	4,319	-62%	1	1	0%	1,890	1,466	-22%	0	0	—
case-07 When debugging a pipeline failure in project `fluid_dynamics`, a developer proposes setting `LOG_LEVEL=DEBUG` in the environment to see maximum diagnostic output. Is `DEBUG` the correct value for maximum verbosity in this pipeline runner?	fail→pass	9,070	3,316	-63%	1	1	0%	1,598	1,414	-12%	0	0	—
case-08 The test suite for project `energy_grid` passed all unit tests, but the pipeline failed right after pytest execution. What target coverage percentage threshold causes pipeline halts if not met?	fail→pass	7,313	5,095	-30%	1	1	0%	1,354	1,808	+34%	0	0	—
case-09 We need to debug structured manuscript metadata and layout before rendering PDFs for project `genetics_core`. Which CLI invocation checks manuscript validity prior to PDF compilation?	fail→pass	10,198	2,984	-71%	1	1	0%	1,757	1,320	-25%	0	0	—
case-10 After generating output files in `output/macro_econ/pdf/`, which CLI tool command validates the generated PDF artifacts?	fail→pass	10,708	1,997	-81%	1	1	0%	1,672	1,109	-34%	0	0	—
case-11 A build failed with obscure TeX font and package errors on a fresh machine. What specific validation utility checks system LaTeX packages?	fail→pass	10,743	1,782	-83%	1	1	0%	2,023	1,070	-47%	0	0	—
case-12 To debug silent failures in project `robotics_lab`, we need maximum verbosity from the execution runner. How should the environment log level be set?	fail→pass	6,168	2,894	-53%	1	1	0%	911	1,294	+42%	0	0	—
case-13 We want to execute only core stages for project `materials_science`, skipping non-core infrastructure checks. Which command flag accomplishes this?	fail→pass	8,749	3,006	-66%	1	1	0%	1,598	1,274	-20%	0	0	—
case-14 Project `bio_cell` is failing in its initial unit test stage. Which specific script file allows direct isolation and execution of stage 1?	fail→pass	9,225	2,546	-72%	1	1	0%	1,357	1,253	-8%	0	0	—
case-15 Where does the pipeline runner pull the canonical list of active projects from when validating project parameter arguments?	fail→fail	11,881	2,156	-82%	1	1	0%	1,897	1,020	-46%	0	0	—
case-16 We spent 3 hours figuring out an obscure environment variable conflict during build debugging for `fin_tech`. What project files should be updated with this non-obvious failure mode?	fail→pass	13,425	4,581	-66%	1	1	0%	2,125	1,601	-25%	0	0	—
case-17 Data corruption was detected deep inside the figure generation routines of project `ocean_depths`. Where in the pipeline flow should the remediation fix be applied?	pass→pass	12,487	4,086	-67%	1	1	0%	1,905	1,441	-24%	0	0	—
case-18 A developer patched a bug in stage 3 of `space_telemetry` and verified that stage 3 passed individually. Can they mark the debugging task as fixed?	pass→pass	8,188	3,395	-59%	1	1	0%	1,285	1,324	+3%	0	0	—
case-19 How do you run tests specifically for project `smart_cities` while enforcing coverage requirements manually?	fail→pass	11,482	3,477	-70%	1	1	0%	1,869	1,260	-33%	0	0	—
case-20 Running the build for `climate_trends` twice in a row produces slightly different floating-point output values in the final table, even though neither run throws an error or fails any stage. How should we debug this nondeterministic drift?	fail→pass	14,366	6,999	-51%	1	1	0%	2,488	2,149	-14%	0	0	—
case-21 The text in section 3 of `econ_policy_doc` claims that GDP increased by 4.2%, but our underlying dataset summary shows 3.8%. Which workflow should we use to audit and verify factual claims stated in manuscript prose?	pass→pass	14,657	2,884	-80%	1	1	0%	2,119	1,284	-39%	0	0	—
case-22 We want to do a general health check and quality audit of our validation infrastructure codebase without having any active build failure or failing stage. Which skill applies?	pass→pass	8,323	1,995	-76%	1	1	0%	1,279	1,105	-14%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 19 counted toward the lift figure. The other 3 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +59 percentage points is the difference between those two pass rates over the 19 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 22.7%
Pass rate with the skill: 81.8%
Cases attempted: 22
Cases counted toward lift: 19
Cases that regressed: 0
Skill version measured: v1
Run completed: 8/4/2026, 00:34 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-04

Prompt

When trying to fix a LaTeX rendering issue in manuscript project `astro_physics`, running the entire build pipeline from start to finish takes 15 minutes each iteration. What command flag pattern should be used to bypass already successful upstream stages while isolating and fixing the renderer?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run82

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns-17% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 54 days ago

Other measured skills in the registry, with their headline benchmark lift.