Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →ashutoshsinghpr7/trace-harness-launch-failure
Empty session traces mean launch failure, not agent behavior — verify api_call_count/tool_call_count and stdout.txt before analyzing.
Passed 13 of 19 test cases with the skill, and 3 of 19 without it.
and 16 more in the full report
11 fixed4 passed either way7 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.
We noticed that our evaluation harness has an invalid default model configuration that causes immediate launch failure, but developers are trying to write prompt engineering proposals based on the current run scores. What is the mandatory immediate action before taking any prompt proposals seriously?
| Checked | Without skill | With skill |
|---|---|---|
| Demands fixing the harness model launch bug before conducting further analysis or adopting prompt proposals. | Passed | Passed |
| Explains that proposals based on empty traces rely on inferred data rather than actual agent execution. | Missed | Passed |
The instructions your model reads when the skill loads.
Four training traces contain ONLY the initial task prompt: 1 message,
0 tool calls, 0 API calls, ~1 s duration. No agent reasoning, tool call, or
final answer anywhere — yet meta.json carries scores (0.0 / 1.0).
Every recorded run died at launch: runs/<task>/stdout.txt shows
"HTTP 400: default is not a valid model ID" — the harness sent an invalid
model ID instead of the configured model, so the agent never executed. The
scores were produced by grading the on-disk deliverables against the
exact-match graders in tasks.json — not from any recorded agent behavior.
api_call_count / tool_call_count / message_count in the trace and
stdout.txt for launch errors. Empty trace == no behavioral evidence.tasks.json → grader.expected), and read spec.md + input data to
explain the score.test cases passed, vs 3 / 19 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.