SIA Harness

Deterministic harness for Meta → Target → Feedback generation loops with public/private task splits. Implements contracts inspired by hexo-ai/sia; does not vendor upstream code.

Commands

bash
uv run python -m infrastructure.sia.cli validate projects/templates/template_sia/tasks/mini_classify
uv run python -m infrastructure.sia.cli inspect-run RUN_SUMMARY_JSON [--json]
uv run python scripts/pipeline/stage_02_analysis.py --project templates/template_sia

Public API

python
from infrastructure.sia import (
    RunConfig,
    TaskLayout,
    load_agent_execution,
    run_evaluation,
    run_sia_loop,
    validate_task_dir,
)

Task layout

Each task directory exposes:

• data/public/ — agent-visible inputs
• data/public/task.md — required task description (validate_task_dir hard-fails if absent)
• data/public/evaluate.py — writes results.json with metric_name, metric_value, n_samples
• data/private/ — evaluation-only labels
• reference/ — baseline target agent (reference/reference_target_agent.py required)

Exemplar project

projects/templates/template_sia/ runs fixture replay by default (live=False). Pass --live-sia on scripts/run_sia_loop.py for opt-in Ollama-backed feedback (not CI) — the feedback note is illustrative, not applied; live mode performs no code mutation and uses no sandbox, so cross-generation improvement is shown only via fixtures.