---
name: ashutoshsinghpr7-trace-harness-launch-failure
description: "Empty session traces mean launch failure, not agent behavior — verify api_call_count/tool_call_count and stdout.txt before analyzing."
license: "MIT"
metadata:
  decimalai-skill-id: "35bf06e2-2fcc-436d-ad3c-fad10819505e"
  decimalai-version-id: "6cfedb24-a7f7-4f61-93a5-58c8d2c8db8e"
  decimalai-version: "1"
  decimalai-body-sha256: "cc93eff6a27030c7aaf5010c5f3e50b11ac7e6d2d3e36d6259f76a43b83ba4cf"
  decimalai-name: "ashutoshsinghpr7/trace-harness-launch-failure"
  decimalai-format: "1"
  decimalai-invocation: "model"
  decimalai-category: "ai-ml"
  decimalai-source: "https://app.decimal.ai/s/ashutoshsinghpr7-trace-harness-launch-failure@1/SKILL.md"
---
# Trace Harness Launch Failure — empty sessions, scores from deliverables

## Problem

Four training traces contain ONLY the initial task prompt: 1 message,
0 tool calls, 0 API calls, ~1 s duration. No agent reasoning, tool call, or
final answer anywhere — yet `meta.json` carries scores (0.0 / 1.0).

## Root cause

Every recorded run died at launch: `runs/<task>/stdout.txt` shows
"HTTP 400: default is not a valid model ID" — the harness sent an invalid
model ID instead of the configured model, so the agent never executed. The
scores were produced by grading the on-disk deliverables against the
exact-match graders in tasks.json — not from any recorded agent behavior.

## Fix

- Before analyzing "agent behavior", verify the session actually ran: check
  `api_call_count` / `tool_call_count` / `message_count` in the trace and
  stdout.txt for launch errors. Empty trace == no behavioral evidence.
- When traces are empty, reconstruct ground truth by diffing the on-disk
  deliverable against the task's grader expectation
  (`tasks.json` → `grader.expected`), and read spec.md + input data to
  explain the score.
- Treat prior-iteration behavioral claims as unverifiable until new traces
  confirm them; flag evidence status in the run log.
- Surface the harness bug so the runner is fixed before the next iteration,
  otherwise every proposal rests on inferred data.