Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →alibaba/custom-engine-e2e-fixture
This directory is a minimal skill-up fixture that exercises the **Custom
Passed 19 of 21 test cases with the skill, and 8 of 21 without it.
and 12 more in the full report
13 fixed9 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.
What is the default shell script filename used as the mock agent executable in the local transport fixture directory?
| Checked | Without skill | With skill |
|---|---|---|
| The response specifies `agent.sh` as the mock agent script name. | Missed | Passed |
The instructions your model reads when the skill loads.
This directory is a minimal skill-up fixture that exercises the Custom
Engine local and http transports end-to-end. It is consumed by
e2e/custom_engine_test.go.
agent.sh + evals/eval.yaml cover the local transport: agent.sh is a
deterministic stand-in for a real custom agent that reads the SessionInput
JSON the framework writes to ${input_file} and emits a fixed SessionResult
on stdout. The case asserts that final_message flows from the agent's stdout
into the report.
evals/eval-http.yaml exercises the http transport against the same
hello.yaml case. TestPipeline_CustomEngine_HTTPTransport starts an
in-process httptest server, points ${CUSTOM_AGENT_ENDPOINT} at it, and
asserts the posted SessionInput reaches the server and the returned
SessionResult flows into the report — so the http stand-in agent lives in the
test rather than in agent.sh.
The eval-stateful-*.yaml fixtures exercise opt-in stateful conversations for
both transports, including session-ID propagation, per-turn post-conditions,
capture, substitution, and a final response that needs no continuation ID.
test cases passed, vs 8 / 21 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.