Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Skill for the Self-Improvement Agent (SIA) harness contract. Use when validating task public/private layouts, generation artifact trees, evaluation runners, fixture replay loops, or opt-in live Meta→Target→Feedback cycles in template projects.
.claude/skills/docxology-infrastructure-sia/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-14 | ✗→✓ | ▲ Improved | -69% | 0% |
| case-02 | ✗→✓ | ▲ Improved | 182% | 0% |
| case-03 | ✗→✓ | ▲ Improved | -41% | 0% |
| case-07 | ✗→✓ | ▲ Improved | -78% | 0% |
| case-08 | ✗→✓ | ▲ Improved | -71% | 0% |
Deterministic harness for Meta → Target → Feedback generation loops with public/private task splits. Implements contracts inspired by hexo-ai/sia; does not vendor upstream code.
bashuv run python -m infrastructure.sia.cli validate projects/templates/template_sia/tasks/mini_classify uv run python -m infrastructure.sia.cli inspect-run RUN_SUMMARY_JSON [--json] uv run python scripts/pipeline/stage_02_analysis.py --project templates/template_sia
pythonfrom infrastructure.sia import ( RunConfig, TaskLayout, load_agent_execution, run_evaluation, run_sia_loop, validate_task_dir, )
Each task directory exposes:
data/public/ — agent-visible inputsdata/public/task.md — required task description (validate_task_dir hard-fails if absent)data/public/evaluate.py — writes results.json with metric_name, metric_value, n_samplesdata/private/ — evaluation-only labelsreference/ — baseline target agent (reference/reference_target_agent.py required)projects/templates/template_sia/ runs fixture replay by default (live=False). Pass --live-sia on scripts/run_sia_loop.py for opt-in Ollama-backed feedback (not CI) — the feedback note is illustrative, not applied; live mode performs no code mutation and uses no sandbox, so cross-generation improvement is shown only via fixtures.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-04 | pass→pass | 4,463 | 2,463 | -45% | 1 | 1 | 0% | 721 | 812 | +13% | 0 | 0 | — |
case-14 | fail→pass | 11,500 | 1,682 | -85% | 1 | 1 | 0% | 2,233 | 692 | -69% | 0 | 0 | — |
case-01 | fail→fail | 6,960 | 5,816 | -16% | 1 | 1 | 0% | 1,439 | 937 | -35% | 0 | 0 | — |
case-02 | fail→pass | 2,052 | 2,753 | +34% | 1 | 1 | 0% | 330 | 931 | +182% | 0 | 0 | — |
case-03 | fail→pass | 5,665 | 3,494 | -38% | 1 | 1 | 0% | 949 | 556 | -41% | 0 | 0 | — |
case-05 | pass→pass | 3,694 | 3,207 | -13% | 1 | 1 | 0% | 682 | 964 | +41% | 0 | 0 | — |
case-06 | pass→pass | 2,604 | 2,787 | +7% | 1 | 1 | 0% | 457 | 898 | +96% | 0 | 0 | — |
case-07 | fail→pass | 33,514 | 1,613 | -95% | 1 | 1 | 0% | 3,226 | 719 | -78% | 0 | 0 | — |
case-08 | fail→pass | 13,207 | 3,599 | -73% | 1 | 1 | 0% | 2,324 | 680 | -71% | 0 | 0 | — |
case-09 | fail→pass | 10,541 | 1,726 | -84% | 1 | 1 | 0% | 1,804 | 680 | -62% | 0 | 0 | — |
case-10 | pass→pass | 16,177 | 1,649 | -90% | 1 | 1 | 0% | 2,818 | 652 | -77% | 0 | 0 | — |
case-11 | fail→pass | 8,575 | 1,458 | -83% | 1 | 1 | 0% | 1,572 | 652 | -59% | 0 | 0 | — |
case-12 | fail→pass | 16,148 | 1,304 | -92% | 1 | 1 | 0% | 2,704 | 614 | -77% | 0 | 0 | — |
case-13 | fail→pass | 12,403 | 1,723 | -86% | 1 | 1 | 0% | 1,785 | 685 | -62% | 0 | 0 | — |
case-15 | fail→pass | 5,907 | 1,307 | -78% | 1 | 1 | 0% | 1,153 | 638 | -45% | 0 | 0 | — |
case-16 | fail→pass | 9,604 | 3,130 | -67% | 1 | 1 | 0% | 1,494 | 698 | -53% | 0 | 0 | — |
case-17 | fail→pass | 12,629 | 2,436 | -81% | 1 | 1 | 0% | 1,898 | 780 | -59% | 0 | 0 | — |
case-18 | fail→pass | 11,029 | 5,315 | -52% | 1 | 1 | 0% | 1,852 | 1,249 | -33% | 0 | 0 | — |
case-19 | pass→pass | 9,351 | 1,892 | -80% | 1 | 1 | 0% | 1,310 | 703 | -46% | 0 | 0 | — |
case-20 | pass→pass | 8,539 | 1,442 | -83% | 1 | 1 | 0% | 1,396 | 623 | -55% | 0 | 0 | — |
case-21 | fail→pass | 9,430 | 1,859 | -80% | 1 | 1 | 0% | 1,618 | 684 | -58% | 0 | 0 | — |
case-22 | fail→pass | 14,454 | 1,994 | -86% | 1 | 1 | 0% | 2,613 | 732 | -72% | 0 | 0 | — |
case-23 | fail→fail | 8,674 | 1,518 | -82% | 1 | 1 | 0% | 1,574 | 604 | -62% | 0 | 0 | — |
case-24 | fail→fail | 10,533 | 1,816 | -83% | 1 | 1 | 0% | 1,779 | 690 | -61% | 0 | 0 | — |
case-25 | fail→pass | 14,814 | 1,590 | -89% | 1 | 1 | 0% | 2,677 | 629 | -77% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 25 cases were attempted. The headline lift of +64 percentage points is the difference between those two pass rates over the 25 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.