Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Use `vibe watch` to run a managed Harness waiter that returns to the same conversation later. Best for reviews, CI, files, logs, and other wait-now-continue-later workflows.
.claude/skills/avibe-bot-background-watch-hook/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-07 | ✗→✓ | ▲ Improved | 226% | 0% |
| case-09 | ✗→✓ | ▲ Improved | 177% | 0% |
| case-10 | ✗→✓ | ▲ Improved | 391% | 0% |
| case-11 | ✗→✓ | ▲ Improved | 572% | 0% |
| case-12 | ✗→✓ | ▲ Improved | 438% | 0% |
Use this skill when the job is "wait now, continue later in the same conversation".
What it gives the agent:
Good trigger scenarios:
Prefer vibe watch when the wait should be inspectable, pausable, resumable, or removable later.
vibe watch addMain entrypoint. Starts a managed background watch and creates a follow-up Agent Run after the waiter succeeds or reaches a terminal failure.
vibe watch list, vibe watch show, vibe watch update, vibe watch pause, vibe watch resume, vibe watch removeUse these to inspect and manage the watch after creation.
scripts/wait_pr.pyBundled GitHub waiter for PR activity, with optional exact-head Actions monitoring.
vibe watch FirstUse vibe watch add first. Most tasks only need:
Generic shape:
bashvibe watch add \ --message "<what the next Agent Run should do>" \ --name "<optional label>" \ -- \ <waiter command ...>
Default behavior:
Use --forever when the same waiter should re-arm after each detected event instead of exiting after one follow-up.
vibe watch Parameters To Remember--message: the instruction template for the follow-up Agent Run created from waiter output--name: optional label for later management--session-id: only when the follow-up should continue a different explicit Agent Session--create-session --same-scope: create a visible sibling Session for the follow-up instead of continuing this conversation--create-session --scope-id <scopes.id>: create the follow-up Session in a specific existing scope--forever: re-arm after each detected event--timeout: per-cycle timeout--lifetime-timeout: whole-watch lifetime cap, mainly for forever watchesManagement commands:
vibe watch listvibe watch show <watch-id>vibe watch update <watch-id> --name '...'vibe watch pause <watch-id>vibe watch resume <watch-id>vibe watch remove <watch-id> hides the watch while keeping prior run historyWrite waiters to follow this contract:
exit 0: event detected; final summary printed to stdoutexit 64 plus the line avibe-watch: no-event on stderr: cycle completed with nothing worth reporting; no follow-up Agent Run, the watch ends (once) or re-arms (--forever)exit 124: timeout; still send a timeout follow-upExit 64 is the token-saving path. Every other terminal exit costs one Agent turn, so a waiter whose normal outcome is uninteresting — green CI, review chatter that was filtered out — should end on 64 rather than reporting "nothing to do". It is a clean ending, so a once watch that retires on 64 reads as completed rather than failed, and whatever the waiter wrote to stderr is logged beside the watch id.
The marker is not optional. 64 is also BSD sysexits EX_USAGE, so a watched command that rejects its own arguments exits with it — and it must keep failing loudly rather than being read as a quiet cycle and, in --forever, rerun indefinitely. A bare 64 is therefore treated as a failure; only 64 with the marker is a no-event cycle. In the bundled waiters, _github_wait_common.no_event("<summary>") prints the summary and the marker to stderr and returns the code, so return no_event(...) is the only place the contract has to be spelled out.
Keep the output split clean:
stdout: final summary for the next turnstderr: polling logs and diagnosticsDelay:
bashvibe watch add \ --name "Delay follow-up" \ --message "The delayed check completed. Continue from the result below." \ -- \ bash -lc 'sleep 120; echo "Timer finished after 120 seconds."'
File appears:
bashvibe watch add \ --name "Wait for export file" \ --message "The export file is ready. Inspect it and continue." \ -- \ bash -lc 'while [ ! -f /tmp/export.json ]; do sleep 10; done; echo "Detected /tmp/export.json"'
Log match:
bashvibe watch add \ --name "Watch app log" \ --message "The expected log pattern appeared. Inspect the event and continue." \ --forever \ -- \ bash -lc 'tail -Fn0 /tmp/app.log | while read -r line; do case "$line" in *READY*) echo "$line"; break;; esac; done'
Use the current Avibe context:
--session-id <id> only when the follow-up should continue a different existing Agent Session.--create-session --same-scope when follow-ups should run in one visible sibling Session under the same Workbench project or IM scope.--forever watches that need a separate visible Session for each event, use --create-session-per-run --same-scope.--create-session --scope-id <scopes.id> when follow-ups should run in one Session under a specific existing scope.--create-session-per-run --scope-id <scopes.id>.--cwd is omitted while creating a Session, Avibe uses the command's current working directory.For vibe watch add:
--timeout is the waiter timeout for one cycle21600 seconds0 means no per-cycle timeout--forever means re-arm after each detected event--retry-exit-code keeps either mode waiting; a once Watch stops after its first event--lifetime-timeout limits the whole long-running watch; default is 0 meaning run until killedThis separation matters: a once Watch may now have several retry cycles, and a forever Watch can still use a bounded timeout for each cycle.
Exit 0 means one new reportable event, never merely that a condition remains true. For a persistent level, return an allowed retry code (default 75) until a new edge is observed. Exit 64 plus avibe-watch: no-event on stderr is a completed cycle with nothing worth reporting. A forever waiter must keep a durable cursor, state transition, or domain cooldown so it cannot emit the same level repeatedly.
Avibe admits only one queued/running follow-up per Watch. A forever Watch re-arms after that Agent Run settles and a five-second safety delay. If the waiter still produces six successful events within 60 seconds, Avibe pauses the Watch and sends the target Agent one repair message containing the bounded latest waiter output. The Agent should inspect and fix the waiter, and resume only after verifying an unambiguous, reversible fix.
This skill ships bundled GitHub waiters:
scripts/wait_pr.pyWaits for GitHub PR review activity, including reviews, inline review comments, PR conversation comments, PR status transitions such as draft -> open, open -> merged, or open -> closed, and the special Codex +1 reaction on the PR body. It can also wait for newly opened PRs in a repository. When one or more --workflow values are provided for a specific PR, the same waiter also watches every matching Actions run at the PR's current head and optional branch. Add --sha only to pin a one-shot wait to one exact head.
scripts/wait_issue.pyWaits for GitHub issue activity, either newly opened issues in a repository or new comments on a single issue.
scripts/wait_action.pyWaits for selected GitHub Actions workflow runs on a specific commit SHA to finish when there is no PR activity stream to combine with them. Workflow failures are reported as an event so the follow-up turn can inspect and handle them.
Use bundled waiters as examples or as ready-to-run building blocks. The main skill is still vibe watch; the waiter is only the thing that blocks until the condition is met. When running a bundled script through uv, prefer uv run --no-project ... so the script does not accidentally attach itself to an unrelated parent project. Bundled GitHub waiters classify temporary network failures and GitHub 408/429/5xx responses as retryable. The one-shot PR and Actions waiters retry those failures inside the same process so the managed watch stays alive. A cycle-oriented waiter may use exit code 75 only when its supervisor explicitly opts into retrying that code. Exit code 64 with the avibe-watch: no-event marker means a cycle finished with nothing worth an Agent turn.
Run bundled waiters relative to the directory containing this loaded SKILL.md. The examples below use BACKGROUND_WATCH_HOOK_DIR for that directory:
bashBACKGROUND_WATCH_HOOK_DIR="<directory containing the loaded SKILL.md>"
For a PR delivery loop, prefer one durable combined wait_pr.py watch. It observes PR review, comment, reaction, thread, lifecycle, and head-change events together with selected Actions workflows for the PR's current head. Use --forever and one state file for the whole loop: when a push changes the head, the next cycle fetches Actions for the new exact SHA without replacing the Watch or rebuilding its PR baseline. Use wait_pr.py without CI arguments for PR-only monitoring. Use wait_action.py only for an Actions wait that is not attached to a PR.
At least one repeatable --workflow enables combined CI monitoring. Omit --sha for the normal delivery loop: each cycle resolves the PR's current head and queries Actions for that exact SHA. --branch, --max-pages, and --success-conclusion remain optional. Add --sha only when the caller intentionally wants a fixed-head one-shot wait. The waiter stays quiet while a requested run is missing or still running, then reports the complete exact-head result when every requested workflow has terminal runs. Every distinct matching run ID is included, so an earlier failed rerun remains visible to the follow-up turn. Set the forever Watch's --timeout 0: the default 21600-second per-cycle timeout treats six quiet hours as a terminal failure, which is not a meaningful end condition for a PR delivery loop.
bashSTATE_FILE="$HOME/.avibe/state/watch-cursors/pr-151-review.json" BRANCH="$(gh pr view 151 --repo avibe-bot/avibe --json headRefName --jq .headRefName)" uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe --pr 151 \ --branch "$BRANCH" \ --workflow lint \ --actionable-only \ --state-file "$STATE_FILE" --seed-state vibe watch add \ --name "Watch PR 151 review and CI" \ --forever \ --timeout 0 \ --message "PR #151 has new review activity, a head change, or current-head CI activity. Fetch the latest PR and Actions state, resolve actionable findings, and leave this durable combined Watch armed until close-out. Summarise the round here in one or two lines; do not post that summary as a PR comment." \ -- \ uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe --pr 151 \ --branch "$BRANCH" \ --workflow lint \ --actionable-only --settle 20 --state-file "$STATE_FILE" --interval 60 \ --timeout 0 # After the seeded Watch is confirmed live, push or post the review trigger. # Keep this same state file and Watch for every later head and review round.
Do not combine --new-prs with CI arguments. A new PR has no stable exact-head gate until the follow-up turn resolves its head and workflow set.
One-shot watch:
bashSTATE_FILE="$HOME/.avibe/state/watch-cursors/pr-151-review.json" uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe --pr 151 --state-file "$STATE_FILE" --seed-state # Push or post the review trigger only after the baseline is durable. vibe watch add \ --name "Watch PR 151 reviews" \ --message "PR #151 has new review activity. Fetch the latest review state and resolve the actionable findings on the PR. Then summarise the round here in one or two lines -- which findings you resolved and what changed -- and do not post that summary as a PR comment. Save a longer message for the review passing, the loop being blocked, or a decision that needs the user." \ -- \ uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe \ --pr 151 \ --actionable-only \ --settle 20 \ --state-file "$STATE_FILE" \ --interval 60
Before the first watched push or review trigger in the delivery loop, seed an owner-specific state file from the current complete PR snapshot. Arm the forever Watch with that exact file and confirm it is live before taking the watched action. Set --timeout 0 on both sides of the -- command separator: the first disables the Watch supervisor's per-cycle deadline, while the second disables the bundled waiter's own default six-hour deadline. GitHub request timeouts remain bounded inside the waiter. A nonzero timeout at either layer is for a Watch whose lack of an event by that deadline is itself reportable. After the Watch starts, never reseed or replace its state between rounds: a later event may already exist, and turning it into the new baseline silently drops it. The forever Watch promotes each delivered batch and compares the next cycle against that durable pre-event snapshot, including activity that landed while the Agent follow-up was running.
Use --catch-up only when deliberately processing historical activity. It is not a substitute for the pre-action baseline in a review loop.
Prefer --actionable-only for review loops. Without it the waiter wakes the Agent for every comment on the PR, including the @codex review triggers the loop itself posts and the bodyless COMMENTED review envelope GitHub wraps around inline comments. With it the waiter still reports inline review comments, reviews carrying a verdict or a body, the Codex pass reaction, and merged/closed transitions — which is everything the review loop needs to make progress or close out.
Narrow it further with --ignore-author <login> and --ignore-comment-pattern <regex> (both repeatable). Filtered items still advance the cursors, so they are examined once and never re-reported. These filters suppress review/comment payloads, not review-thread status. A thread becoming unresolved or resolved remains an independent wake signal because thread state is a separate mutable resource and may later be changed by a different actor.
--settle <seconds> is worth setting on any review loop. A bot review arrives as a burst of inline comments plus an envelope, so the poll that happens to catch the first fragment would otherwise report it alone and the rest would arrive as a second event. With --settle the waiter re-polls until the set stops growing (at most three extra polls) and reports the whole batch as one event, which is one Agent turn instead of several. 20 seconds is a reasonable starting point. The window never runs past --timeout: settling is skipped unless both the wait and a full re-poll fit in what is left of the deadline, so a batch already worth a turn is reported rather than lost to the timeout kill. Keep --settle well under --timeout.
Catch up on existing activity first:
bashSTATE_FILE="$HOME/.avibe/state/watch-cursors/pr-151-catch-up.json" vibe watch add \ --name "Catch up PR 151 reviews" \ --message "PR #151 already has review activity. Fetch the latest review state and resolve the actionable findings on the PR. Then summarise the round here in one or two lines -- which findings you resolved and what changed -- and do not post that summary as a PR comment. Save a longer message for the review passing, the loop being blocked, or a decision that needs the user." \ -- \ uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe \ --pr 151 \ --state-file "$STATE_FILE" \ --catch-up
Stay armed for future activity:
bashSTATE_FILE="$HOME/.avibe/state/watch-cursors/pr-151-forever.json" uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe --pr 151 --state-file "$STATE_FILE" --seed-state vibe watch add \ --name "Monitor PR 151 reviews" \ --forever \ --timeout 0 \ --lifetime-timeout 0 \ --message "PR #151 has new review activity. Fetch the latest review state and resolve the actionable findings on the PR. Then summarise the round here in one or two lines -- which findings you resolved and what changed -- and do not post that summary as a PR comment. Save a longer message for the review passing, the loop being blocked, or a decision that needs the user." \ -- \ uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe \ --pr 151 \ --actionable-only \ --settle 20 \ --state-file "$STATE_FILE" \ --interval 60 \ --timeout 0
Always pass --state-file to a --forever watch. Each cycle is a fresh waiter process, so without it the next cycle re-snapshots the PR as its baseline and anything that arrived between the previous cycle's exit and that snapshot is lost. The file carries the resolved GitHub login and the complete mutable PR baseline forward. PR cycles reread complete review/comment/thread collections because edits, deletions, and thread-resolution changes are wake-worthy state. For an ordinary wait, that normalized complete snapshot is the single wake/no-wake decision. Numeric cursors, fingerprints, and the thread map only describe a detected change; they cannot wake independently. --catch-up and explicit --since-*-id flags are the deliberate replay modes and therefore remain cursor-driven. A legacy state file with cursors but no complete snapshot is rejected until it is deliberately caught up or reseeded, rather than silently absorbing mutable changes. The login is reused only while the token still fingerprints to the account it was resolved for. Explicit cursor flags still request a replay from that cursor, while the complete PR collections remain the source of truth for edits and removals. The state also records the last observed PR head, review/comment fingerprints, and every review-thread resolution state, so a pushed head, edited object, deletion, or thread transition is activity even when no new numeric ID exists. Cursors that cover a reported event are not committed by the cycle that reports it. A waiter cannot observe its own delivery — vibe watch reads its stdout only after the process exits — so those cursors are staged under pending while the committed ones stay before the event, together with the rendered report and the value of AVIBE_WATCH_LAST_DELIVERY the cycle started from. That variable is when this watch last had a report durably queued, stamped in the same transaction as the follow-up, so any later cycle that reads a different value knows the report was delivered and promotes the staged cursors. An unchanged value means it may never have been queued, so they are dropped and the event is reported again: at-least-once, costing one repeated Agent turn instead of losing the activity for good. Comparing a durable stamp rather than consuming a one-shot acknowledgement is what makes this correct across a service restart, and for a once watch, whose one report is followed by no cycle at all until the user resumes it. A manual run has no supervisor, and there printing is the delivery, so it commits straight after reporting. Progress made by filtering — new activity that was deliberately not reported — commits immediately either way, since there is no delivery to wait for. Three failures are terminal rather than a warning — a path that cannot be written to (checked before the first poll), a path already owned by another watch, and an existing file whose cursors cannot be read — and each stops the watch with exit 1 rather than polling on without the cursors it was asked to keep, or clobbering another watch's. A corrupt or unrecognised state file is left exactly as found: re-baselining from the current PR would skip everything that arrived after the cursor it did hold and then overwrite the only evidence of how far the watch had got. An empty file is the one exception, since it is a claim caught between its exclusive create and its first write and never held a cursor at all.
Ownership is claimed, not assumed. A missing state file is created before the first poll holding nothing but the identity it belongs to, so two watches started together cannot both see an unowned path; the one that loses that exclusive create reads the winner's claim and stops. Ownership is re-checked before every replacement as well, because the loser of a microsecond-wide race has already passed the startup check. Identity is the repo, the PR, and the options that decide what the watch reports (--actionable-only, --ignore-author, --ignore-comment-pattern, --include-self-comments, --new-prs) — two watches on the same PR that report different things cannot share cursors either, because the filtered one advances past events the other never reported. Pacing options such as --interval and --settle are not part of it. When vibe watch runs the cycle it also names the watch in AVIBE_WATCH_ID, and the waiter records it as the owner, so even two watches configured identically down to the last filter are kept apart; a manual run has no id and adopts whatever it finds -- and a managed watch that starts on a file with no owner, from a manual run or an older version, stamps itself on it before polling, because an owner that is absent fits every watch and two of them would share the path. That decision is read-then-write, so it is taken under a <state-file>.lock sidecar; the lock lives beside the state file rather than on it, since the state file is replaced rather than rewritten. Give each watch its own state file.
Arguments are validated before any of this: a rejected --ignore-comment-pattern or a missing token exits 2 without claiming the path, so the corrected re-run is not refused as a different watch's state.
GitHub-specific notes:
--catch-up reports activity that already exists at startup, and overrides savedcursors when a state file is present; an explicit --since-*-id still wins
--catch-up or a --state-file, the waiter snapshots current PR activity as the baselineso edits and deletions remain observable, reactions are filtered server-side with content=+1, and unchanged pages revalidate to 304, which GitHub does not charge against the rate limit — an idle watch can poll for hours for free
chatgpt-codex-connector orchatgpt-codex-connector[bot] leaves a +1 reaction on the PR body instead of posting a comment; pass reactions remain visible even when --event-limit is reached
loop asleep
--include-self-comments to keep themNew PRs in a repository:
bashSTATE_FILE="$HOME/.avibe/state/watch-cursors/new-prs-avibe.json" uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe --new-prs --state-file "$STATE_FILE" --seed-state vibe watch add \ --name "Watch new PRs" \ --message "The repository has new pull requests. Review the new PRs and continue as needed." \ -- \ uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_pr.py" \ --repo avibe-bot/avibe \ --new-prs \ --state-file "$STATE_FILE" \ --interval 60
New issues or issue comments:
bashuv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_issue.py" --repo avibe-bot/avibe --new-issues --interval 60 uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_issue.py" --repo avibe-bot/avibe --issue 157 --interval 60
Standalone GitHub Actions for a pushed commit that is not being monitored through a PR:
bashvibe watch add \ --name "Watch CI" \ --message "GitHub Actions failed. Inspect the result below and fix the failures. Then summarise the round here in one or two lines -- what failed and what you changed. Save a longer message for the build going green, the loop being blocked, or a decision that needs the user." \ -- \ uv run --no-project "$BACKGROUND_WATCH_HOOK_DIR/scripts/wait_action.py" \ --repo cyhhao/sub2api \ --branch main \ --sha "$HEAD_SHA" \ --workflow CI \ --workflow "Security Scan" \ --interval 60
For a merge gate, omit --only-on-failure: a successful exact-head build must wake the Agent so it can perform the final gate and close out the watch. Use --only-on-failure only when a green result intentionally needs no follow-up; that mode exits 64 with the no-event marker and records the summary in the Avibe watch log instead of creating an Agent turn.
watching a loop they cannot see inside: with no per-round summary all they get is a stream of tool activity, and no way to tell a converging PR from one that is thrashing on the same finding. Name the findings resolved and what changed, not a diff walkthrough — the detail is already in the commit and on the PR. Keep it in the conversation only; re-posting the same summary as a PR comment adds noise to the review the bot is reading. Save a longer, user-facing message for the outcomes a human has to act on: the review passed, the loop is blocked, or a decision has to be made. A reply containing only a <silent>...</silent> block sends nothing at all, so keep that for cycles that genuinely produced nothing worth reading — a re-poll that found no new findings — not for the rounds that did the work.
vibe watch add, read vibe watch add --help first; the help text explains both argument syntax and runtime behavior such as how --message and waiter stdout become the follow-up Agent Run input.vibe watch over ad-hoc detached shells when the wait should survive the current turn cleanly.| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 14,621 | 9,796 | -33% | 1 | 1 | 0% | 1,611 | 7,583 | +371% | 0 | 0 | — |
case-02 | fail→fail | 18,624 | 11,302 | -39% | 1 | 1 | 0% | 3,090 | 7,648 | +148% | 0 | 0 | — |
case-03 | fail→fail | 9,717 | 7,642 | -21% | 1 | 1 | 0% | 1,577 | 7,195 | +356% | 0 | 0 | — |
case-04 | pass→pass | 4,704 | 16,590 | +253% | 1 | 1 | 0% | 982 | 9,130 | +830% | 0 | 0 | — |
case-05 | pass→pass | 12,110 | 13,175 | +9% | 1 | 1 | 0% | 2,411 | 9,500 | +294% | 0 | 0 | — |
case-06 | fail→fail | 2,945 | 7,834 | +166% | 1 | 1 | 0% | 419 | 7,218 | +1623% | 0 | 0 | — |
case-07 | fail→pass | 15,110 | 4,116 | -73% | 1 | 1 | 0% | 2,342 | 7,633 | +226% | 0 | 0 | — |
case-08 | pass→pass | 8,957 | 3,608 | -60% | 1 | 1 | 0% | 1,722 | 7,423 | +331% | 0 | 0 | — |
case-09 | fail→pass | 15,785 | 18,190 | +15% | 1 | 1 | 0% | 2,966 | 8,228 | +177% | 0 | 0 | — |
case-10 | fail→pass | 23,499 | 2,916 | -88% | 1 | 1 | 0% | 1,494 | 7,342 | +391% | 0 | 0 | — |
case-11 | fail→pass | 25,238 | 4,565 | -82% | 1 | 1 | 0% | 1,152 | 7,742 | +572% | 0 | 0 | — |
case-20 | pass→pass | 6,027 | 2,069 | -66% | 1 | 1 | 0% | 752 | 7,158 | +852% | 0 | 0 | — |
case-12 | fail→pass | 17,074 | 4,093 | -76% | 1 | 1 | 0% | 1,406 | 7,571 | +438% | 0 | 0 | — |
case-13 | fail→fail | 12,514 | 4,806 | -62% | 1 | 1 | 0% | 1,878 | 7,390 | +294% | 0 | 0 | — |
case-14 | fail→pass | 16,922 | 3,414 | -80% | 1 | 1 | 0% | 2,878 | 7,503 | +161% | 0 | 0 | — |
case-15 | pass→pass | 8,957 | 4,446 | -50% | 1 | 1 | 0% | 1,323 | 7,520 | +468% | 0 | 0 | — |
case-16 | pass→pass | 3,704 | 3,434 | -7% | 1 | 1 | 0% | 601 | 7,510 | +1150% | 0 | 0 | — |
case-17 | fail→pass | 5,735 | 3,085 | -46% | 1 | 1 | 0% | 803 | 7,262 | +804% | 0 | 0 | — |
case-18 | fail→pass | 9,508 | 2,944 | -69% | 1 | 1 | 0% | 1,639 | 7,314 | +346% | 0 | 0 | — |
case-19 | fail→pass | 9,069 | 2,360 | -74% | 1 | 1 | 0% | 1,324 | 7,232 | +446% | 0 | 0 | — |
case-21 | fail→pass | 8,872 | 13,422 | +51% | 1 | 1 | 0% | 1,666 | 8,642 | +419% | 0 | 0 | — |
case-22 | fail→fail | 16,912 | 6,360 | -62% | 1 | 1 | 0% | 3,056 | 7,196 | +135% | 0 | 0 | — |
case-23 | fail→pass | 10,764 | 2,024 | -81% | 1 | 1 | 0% | 1,809 | 7,131 | +294% | 0 | 0 | — |
case-24 | fail→pass | 7,562 | 2,724 | -64% | 1 | 1 | 0% | 1,248 | 7,279 | +483% | 0 | 0 | — |
case-25 | pass→pass | 8,314 | 2,725 | -67% | 1 | 1 | 0% | 1,258 | 7,262 | +477% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 25 cases were attempted, and 18 counted toward the lift figure. The other 7 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +48 percentage points is the difference between those two pass rates over the 18 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.