Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Freshly judge exact subject content against
.claude/skills/boshu2-validate/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-06 | ✗→✓ | ▲ Improved | 148% | 0% |
| case-02 | ✗→✓ | ▲ Improved | 52% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 34% | 0% |
| case-04 | ✗→✓ | ▲ Improved | 66% | 0% |
| case-07 | ✗→✓ | ▲ Improved | 43% | 0% |
Independently judge one exact subject against the acceptance in its existing bead or caller source, return one semantic result, and stop. Validate is the sole verdict.v2 writer when persistence is requested. It never asks the model to reconstruct Plan or Candidate packets.
least one entry, and store-verdict refuses an empty one. Plans, audits, reviews, and other control artifacts are not completion subjects unless the caller explicitly requested document review.
content-addressed snapshot; its acceptance digest is derived automatically.
source: runtime | caller and anattester identity.
Missing, colliding, or unattested identities produce NOT_PROVEN. This is a declared trust fact, not cryptographic proof that contexts were isolated.
A caller may request that the fresh validator run on a different model than the author. Dispatch via the controller-session recipe in the agent-native model-dispatch recipe (codex-exec and/or ntm, probed at runtime). Record author and validator model_identity in evidence refs and freshness attestation notes — do not change verdict.v2 schema. If the requested validator model has no live adapter, disclose the unsatisfied diversity request and proceed same-model; never invoke claude -p / claude --print. Single fresh validator remains the default shape.
Before running any acceptance-listed command, classify it as read-only or subject-mutating. Regen scripts, sync scripts, formatters, and anything with --force are subject-mutating until proven otherwise. Never run a subject-mutating check against an uncommitted subject: on 2026-07-15, scripts/test-ci-deterministic-gates.sh regenerated skills-codex/ from HEAD mid-validation and destroyed the uncommitted subject, forcing NOT_PROVEN (verdict b6e759dd...cb6a); only restoring the subject and revalidating in a fresh context produced the PASS (e9b6cdb8...37b9). If a mutating check is genuinely required by acceptance, run it against a disposable copy or a committed subject, never the judged working tree.
not_checked has exactly one meaning: in-scope acceptance surface this validation did not verify. PASS asserts that the whole declared acceptance surface was verified, so a PASS carries no not_checked entries; the helper refuses one and records a validate.integrity finding.
That rule never pays for deleting an honest caveat, because every kind of scope limit has a home that survives inside a PASS:
| Scope limit | Home | Example | |---|---|---| | A criterion proven by a bounded check | criteria[].reason on that criterion | "proven by the unit suite; the full integration matrix was not replayed" | | A declared non-goal or out-of-scope area | the intent source's non-goals, optionally restated as an evidence-backed boundary criterion in criteria | "cli/** is a declared non-goal; the diff proves it untouched" | | Residual risk or judgment caveat | the caller-facing report | "the migration path is untested against pre-3.0 stores" | | Acceptance that genuinely went unverified | not_checked, and the result is NOT_PROVEN rather than PASS | "criterion 3 needs hardware this context cannot reach" |
Emptying not_checked to obtain PASS is a contract violation, not a workaround. If acceptance really went unverified, the honest result is NOT_PROVEN. If the entry was never acceptance in the first place, it belongs in one of the other homes, where it stays visible in the stored artifact instead of being deleted.
The helper ships beside this file. Invoke it through this skill's own directory rather than a checkout-relative path: $SKILL_DIR is the directory containing this SKILL.md — skills/validate/ in a repository checkout, .agents/skills/validate/ in an installed runtime.
| Command | Required | Optional | |---|---|---| | manifest | --root <dir>, --include <path> (repeatable, at least one) | --exclude <path-or-glob> (repeatable), --base-manifest <file>, --git-metadata-json <json>, --output <file> | | verify-manifest | --root <dir>, --manifest <file> | --base-manifest <file> | | snapshot-intent | --source <file> (- reads stdin) | --workspace <dir>, --intent-dir <dir> | | digest | <json-file> positional | none | | store-verdict | --draft, --intent-source, --subject-manifest, --author-context-id, --validator-context-id, --freshness-source <runtime\|caller>, --freshness-attester-id, --scope-result <PASS\|FAIL\|NOT_PROVEN> | --workspace <dir>, --verdict-dir <dir> |
shpython3 "$SKILL_DIR/scripts/validate.py" manifest \ --root . --include skills/validate --exclude '**/*.log' --output manifest.json
subject-manifest.v1 with the manifest commandabove (--root plus at least one --include). The helper uses only filesystem content; Git commit/tree IDs are optional metadata. Derive the manifest at the start of validation and re-derive it at the end; any mismatch between the two is subject mutation and returns NOT_PROVEN.
the subject changed or complete changed-path coverage cannot be derived, return NOT_PROVEN.
runtime-derived actual changed paths against the intent's scope classes. A proven out-of-scope path returns FAIL; incomplete scope evidence returns NOT_PROVEN.
claims, not evidence: re-execute the claimed proofs that bear on acceptance (see the freshness rules below for when a digest-bound receipt suffices). If the subject changes a test, gate, fixture, golden, tolerance, suppression, or acceptance source, determine whether the original intent requires that change and whether green came from implemented behavior rather than a weakened oracle. Green obtained by weakening acceptance is FAIL, not evidence of completion. Judge every acceptance criterion and record criterion-level results, findings, evidence references, checked, and any acceptance surface that went unverified in not_checked (see Scope disclosure).
PASS, FAIL, or NOT_PROVEN. Returnit with criterion results, findings, evidence references, checked, not_checked, the acceptance and subject identities, distinct author and validator context IDs, and the freshness attestation. PASS requires distinct identities, explicit freshness, nonempty checked scope, top-level evidence, evidence for every criterion, and an empty not_checked; route bounded proofs, declared non-goals, and residual risk to the homes named in Scope disclosure rather than deleting them or downgrading a proven result.
downstream consumer requires it, persist canonical verdict.v2 with the helper's store-verdict --draft <draft.json> --intent-source <resolved-intent> --subject-manifest <manifest.json> --author-context-id <id> --validator-context-id <id> --freshness-source <runtime|caller> --freshness-attester-id <id> --scope-result <PASS|FAIL|NOT_PROVEN>. The helper snapshots the exact resolved intent under <workspace>/.agents/ao/intents/sha256/<digest>.intent, then computes and injects intent and subject digests plus author, validator, and freshness facts. Identity and changed-path facts come from runtime-derived inputs and receipts, not model transcription. Storage defaults to <workspace>/.agents/ao/verdicts/sha256/<digest>.json; callers may provide verdict_dir.
Stop.
The digest is SHA-256 over canonical JSON with artifact_digest omitted. Writes use a same-directory temporary file, flush, fsync, and atomic rename. Identical existing content is idempotent success; conflicting content is an integrity failure represented by NOT_PROVEN.
Fresh validation means independent judgment over the exact subject. It does not require mechanically replaying every author command. Verify intent identity, scope, evidence digests, and every acceptance criterion; independently rerun the risk-critical, uncertain, or insufficiently evidenced checks. A digest-bound deterministic receipt may prove routine facts. Replay an expensive full suite only when acceptance requires that result or the supplied receipt cannot establish it.
Validate emits no WARN, confidence, disposition, briefing learning, owner, next action, repair, retry, replan, helper, escalation, tracker, Git, release, closure, or delivery state. Generic provenance may record a verdict later, but ledger availability cannot change its validity.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-05 | pass→pass | 5,327 | 2,643 | -50% | 1 | 1 | 0% | 784 | 2,683 | +242% | 0 | 0 | — |
case-06 | fail→pass | 7,096 | 2,320 | -67% | 1 | 1 | 0% | 1,044 | 2,584 | +148% | 0 | 0 | — |
case-01 | fail→fail | 5,091 | 6,264 | +23% | 1 | 1 | 0% | 292 | 2,497 | +755% | 0 | 0 | — |
case-02 | fail→pass | 12,103 | 3,427 | -72% | 1 | 1 | 0% | 1,913 | 2,911 | +52% | 0 | 0 | — |
case-03 | fail→pass | 13,215 | 3,290 | -75% | 1 | 1 | 0% | 2,102 | 2,821 | +34% | 0 | 0 | — |
case-04 | fail→pass | 10,194 | 2,090 | -79% | 1 | 1 | 0% | 1,532 | 2,549 | +66% | 0 | 0 | — |
case-07 | fail→pass | 13,406 | 3,724 | -72% | 1 | 1 | 0% | 2,006 | 2,862 | +43% | 0 | 0 | — |
case-08 | fail→pass | 11,295 | 6,191 | -45% | 1 | 1 | 0% | 1,830 | 3,407 | +86% | 0 | 0 | — |
case-09 | fail→pass | 6,228 | 2,079 | -67% | 1 | 1 | 0% | 968 | 2,598 | +168% | 0 | 0 | — |
case-10 | fail→pass | 12,284 | 3,958 | -68% | 1 | 1 | 0% | 2,009 | 2,807 | +40% | 0 | 0 | — |
case-11 | fail→pass | 13,145 | 3,561 | -73% | 1 | 1 | 0% | 2,071 | 2,825 | +36% | 0 | 0 | — |
case-12 | pass→pass | 11,407 | 2,512 | -78% | 1 | 1 | 0% | 1,660 | 2,679 | +61% | 0 | 0 | — |
case-13 | pass→pass | 10,361 | 3,178 | -69% | 1 | 1 | 0% | 1,597 | 2,777 | +74% | 0 | 0 | — |
case-14 | pass→pass | 11,371 | 3,299 | -71% | 1 | 1 | 0% | 1,899 | 2,863 | +51% | 0 | 0 | — |
case-15 | fail→pass | 12,074 | 3,652 | -70% | 1 | 1 | 0% | 1,847 | 2,852 | +54% | 0 | 0 | — |
case-16 | pass→pass | 8,642 | 3,895 | -55% | 1 | 1 | 0% | 1,347 | 2,828 | +110% | 0 | 0 | — |
case-17 | pass→pass | 14,759 | 4,472 | -70% | 1 | 1 | 0% | 2,400 | 2,864 | +19% | 0 | 0 | — |
case-18 | pass→pass | 5,428 | 1,625 | -70% | 1 | 1 | 0% | 877 | 2,476 | +182% | 0 | 0 | — |
case-19 | pass→pass | 12,553 | 3,336 | -73% | 1 | 1 | 0% | 1,902 | 2,751 | +45% | 0 | 0 | — |
case-20 | fail→pass | 6,608 | 5,042 | -24% | 1 | 1 | 0% | 1,240 | 3,085 | +149% | 0 | 0 | — |
case-21 | fail→pass | 10,601 | 13,272 | +25% | 1 | 1 | 0% | 1,867 | 3,749 | +101% | 0 | 0 | — |
case-22 | fail→pass | 5,269 | 4,948 | -6% | 1 | 1 | 0% | 825 | 2,988 | +262% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 21 counted toward the lift figure. The other 1 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +59 percentage points is the difference between those two pass rates over the 21 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.