Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

Confidence Scoring · yogsoth-ai | DecimalAI

researchby yogsoth-ai

Confidence Scoring

yogsoth-ai/confidence-scoring

Source yogsoth-ai/de-anthropocentric-research-engine ↗License Apache-2.0

SOP for assigning calibrated confidence scores to causal claims based on evidence quality and quantity.

SkillScore68Maintained

Automatic

Research pack

Last verified 8/22/2026

What it changes

Passed 13 of 19 test cases with the skill, and 6 of 19 without it.

Without skill32%

With skill68%

Checks it passed that the model alone missed

The confidence score is capped at 1.0.
The final confidence score equals 0.6.
The file claims/creatine-muscle.md is modified.

and 13 more in the full report

10 fixed5 passed either way5 still failing2 broke

Pass-rate lift: +36 pp
Tokens: +38%
Agent turns: 0%
Response time: +40%

Model gemini-3.6-flashMeasured 8/22/2026v115 of 22 cases passed with skill · 19 comparable

Tokens, turns and time describe resource use, not a dollar price.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/yogsoth-ai-confidence-scoring/SKILL.md https://app.decimal.ai/s/yogsoth-ai-confidence-scoring/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add yogsoth-ai-confidence-scoring, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/yogsoth-ai-confidence-scoring/ and pins it in .decimal/skills.lock

decimalai skills add yogsoth-ai-confidence-scoring

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/yogsoth-ai-confidence-scoring/SKILL.md https://app.decimal.ai/s/yogsoth-ai-confidence-scoring/SKILL.md

Raw file: /s/yogsoth-ai-confidence-scoring/SKILL.md ↗ · machine-readable summary at /s/yogsoth-ai-confidence-scoring.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 9

Prompt

Analyze the causal claim in `claims/sleep-memory.md` that 8 hours of sleep improves memory consolidation. Supporting evidence weight is 7.0, contradicting weight is 3.0, and strong mechanism exists (+0.1). Write the frontmatter update and return the outcome.

What was checked, without and with the skill
Checked	Without skill	With skill
The returned JSON object includes the string key 'claim'.	Missed	Passed
The confidence score in the returned JSON is set to 0.8.	Missed	Passed

case-13 · gemini-3.6-flash · 8/22/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md25 linessingle file

Confidence Scoring

Assign calibrated confidence scores to causal claims.

Tool

CC file edit

Protocol

Count supporting evidence (weighted by evidence hierarchy)
Count contradicting evidence (weighted similarly)
Calculate net confidence: support_weight / (support_weight + contradict_weight)
Adjust for mechanism plausibility (+0.1 if strong mechanism, -0.1 if no mechanism)
Update claim page frontmatter confidence field

HARD-GATE

Yield

Returns: { claim: string, confidence: number, evidence_count: number }

Track record

Benchmark

13 / 19

test cases passed, vs 6 / 19 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

yogsoth-ai / de-anthropocentric-research-engine

skills/confidence-scoring/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 8/3/2026Version v1

Other skills for the same job

Published skills whose descriptions closely match this one, with their headline benchmark lift.

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Confidence Scoring

Tool

Protocol

HARD-GATE

Yield

Track record

Benchmark

Adoption

SkillSafety

Source

Other skills for the same job

Related skills