Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →asymptote-labs/self-verify-beacon-in-sandbox
Verify a Beacon change end to end by running a real Claude Code session inside a disposable Linux cloud sandbox and checking that Beacon captured what the agent actually did.
Passed 18 of 21 test cases with the skill, and 5 of 21 without it.
and 13 more in the full report
13 fixed5 passed either way3 still failing0 broke2 errored
Tokens, turns and time describe resource use, not a dollar price.
One of 13 test cases the skill fixed. Same model, same prompt, with and without the skill.
A scenario run failed once, but I suspect it might be non-deterministic network noise. What flag does the sandbox runner offer to help differentiate flaky runs from broken code?
| Checked | Without skill | With skill |
|---|---|---|
| Recommends using the --repeat flag to execute scenario runs multiple times. | Missed | Passed |
| Explains that repeated scenario runs help distinguish non-deterministic test flakiness from code defects. | Passed | Passed |
The instructions your model reads when the skill loads.
This repository ships beacon-sandbox, which rents a disposable Linux sandbox from Modal,
installs the Beacon under test, runs a real Claude Code session inside it, and checks whether
Beacon recorded what the agent did.
Read beacon-sandbox/AGENTS.md for the full operating manual — scenario selection, how to
interpret a verdict, and the failure modes that look like Beacon bugs but are not. The commands
below are enough to start; that file is what stops you misreading the result.
Always start here. It is free, needs no sandbox, and prints the exact fix for anything missing:
cd beacon-sandbox
go run ./cmd/beacon-sandbox doctor
Add --json for machine-readable output with a top-level ready boolean.
Apply the fix: line it prints, then rerun doctor. The two build artifacts you can resolve
yourself:
cd cli/beacon && make build-linux-amd64 # if beacon_binary FAILs
cd beacon-sandbox && go run ./cmd/beacon-sandbox doctor --fix # downloads the collector
Two prerequisites you must NOT try to resolve yourself:
modal token new completes through an authenticated web session,
so it opens a browser and blocks — running it yourself will hang until timeout. If
modal_auth FAILs, ask the user to run it, suggesting they type
! pip install modal && modal token new so the output lands in the conversation. In CI,
MODAL_TOKEN_ID and MODAL_TOKEN_SECRET work non-interactively instead.anthropic_credential
FAILs, ask the user which of the three paths they want: ANTHROPIC_API_KEY,test cases passed, vs 5 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.