Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Conformance loop · fallow-rs | DecimalAI

testingby fallow-rs

Conformance loop

fallow-rs/conformance-loop

Source fallow-rs/fallow ↗License MIT

Iteratively improve Fallow analysis accuracy by comparing it with competing tools and verified source truth across a stable real-world corpus.

SkillScore83Maintained

Automatic

QA & Test Engineering pack

Last verified 9/28/2026

What it changes

Passed 17 of 20 test cases with the skill, and 7 of 20 without it.

Without skill33%

With skill83%

Checks it passed that the model alone missed

Specifies executing review as the final step.
Rejects custom categories like Tool Bug or Parser Error.
Directs classifying the discrepancy as a model difference.

and 16 more in the full report

13 fixed7 passed either way3 still failing1 broke

Pass-rate lift: +50 pp
Tokens: -48%
Agent turns: 0%
Response time: -59%

Model gemini-3.6-flashMeasured 9/28/2026v420 of 24 cases passed with skill · 20 comparable

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/fallow-rs-conformance-loop/SKILL.md https://app.decimal.ai/s/fallow-rs-conformance-loop/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add fallow-rs-conformance-loop, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/fallow-rs-conformance-loop/ and pins it in .decimal/skills.lock

decimalai skills add fallow-rs-conformance-loop

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/fallow-rs-conformance-loop/SKILL.md https://app.decimal.ai/s/fallow-rs-conformance-loop/SKILL.md

Raw file: /s/fallow-rs-conformance-loop/SKILL.md ↗ · machine-readable summary at /s/fallow-rs-conformance-loop.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 13

Prompt

Once a conformance fix has been implemented, validated across the corpus with net improvements, and confirmed to pass all tests, what is the required final command/step to complete the process?

What was checked, without and with the skill
Checked	Without skill	With skill
Specifies executing review as the final step.	Missed	Passed

case-10 · gemini-3.6-flash · 9/28/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md17 linessingle file

Conformance loop

Define the capability and stable project corpus.
Run Fallow and comparison tools with documented equivalent settings.
Manually verify disagreements against source.
Classify true positive, false positive, false negative, or model difference.
Record the classification in tests/conformance/adjudications.json before writing the fix. A verdict that lives only in a session or in .plans/ is lost, and the daily lane then re-reports the same disagreement as unadjudicated forever.
Implement one general correction with a regression fixture.
Re-run the full corpus and retain only net improvements.
Run review.

Competitor output is a lead, not ground truth.

Track record

Benchmark

17 / 20

test cases passed, vs 7 / 20 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v5 — 9/15/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

fallow-rs / fallow

.agents/skills/conformance-loop/SKILL.md

View on GitHub ↗

License MITImported 9/15/2026Version v4

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Conformance loop

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills