Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →davepoon/check
Run CIAgent regression checks after changing an AI agent's code, prompts, or knowledge base in a repo that has agentci_spec.yaml, and interpret the results. Use after editing agent logic, before committing agent changes, or when the user asks whether the agent still works.
Passed 19 of 21 test cases with the skill, and 9 of 21 without it.
and 11 more in the full report
11 fixed10 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.
I've added new documentation files into our knowledge base directory. Please run the evaluation checks needed for knowledge updates and summarize the outcome in a couple of sentences detailing the command used, score, and any failures.
| Checked | Without skill | With skill |
|---|---|---|
| The response specifies executing `ciagent generate-checks --dry-run`. | Missed | Passed |
The instructions your model reads when the skill loads.
The repo has agentci_spec.yaml (if it does not, use the onboard skill
instead). Your job: run the right check for the change that was just made,
read the result correctly, and never paper over a failure.
| Situation | Command |
|---|---|
| Spec or wiring changed, or no API keys | ciagent test --mock |
| Agent code / prompt / retrieval changed | ciagent test --yes --format json |
| Result differs from last run, or flakiness suspected | ciagent test --runs 3 --yes |
| Knowledge base changed | ciagent generate-checks --dry-run, review, then apply |
| The LLM judge's verdicts look wrong | ciagent judge-audit |
Live runs (test without --mock, judge-audit, generate-checks) call model
APIs on the user's keys. Mock mode is free. If the user has not already
approved live runs in this session, prefer --mock or ask.
Exit codes: 0 pass (including flaky-but-passing), 1 correctness failure
(with --runs N: failed in every run), 2 infra or config error — fix the
setup, not the agent.
With --format json: per-query entries carry layer results (correctness /
path / cost) and the answer text; with --runs N a top-level stability block
lists flipped queries with flip_source.
Flip sources route the work:
agent-variance — the agent's answer changed between runs → fix the agent
(prompt, retrieval, temperature).judge-flake — same answer, the LLM judge changed its verdict → fix the
eval (tighten the rubric or replace with a deterministic check).infra-error — a judge API call failed → retry; fix nothing.mixed — ambiguous; look at the answers yourself.test cases passed, vs 9 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.