Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →borghei/batch-api-orchestrator
This skill should be used when the user asks to "batch LLM requests", "should I use the batch API", "estimate batch vs realtime cost", "design a bulk LLM job", or "process thousands of prompts cheaply".
Passed 11 of 22 test cases with the skill, and 11 of 22 without it.
and 1 more in the full report
4 fixed16 passed either way3 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
Which flag configures backoff handling in `scripts/batch_job_planner.py`?
| Checked | Without skill | With skill |
|---|---|---|
| Specifies the `--retry-policy` argument flag | Missed | Passed |
The instructions your model reads when the skill loads.
Category: Engineering Domain: AI Engineering
Decide when to run LLM work through an asynchronous batch API versus realtime/streaming, then design the job so it is cheap, idempotent, and resilient to partial failure. Batch APIs typically cost roughly half of realtime in exchange for higher latency (results arrive over minutes to hours, not milliseconds), which makes them ideal for evals, backfills, embeddings, and bulk classification/extraction — and wrong for anything a human is waiting on. This skill is model- and vendor-agnostic: it reasons about the batch pattern, not any one provider's API.
Before recommending or designing a batch job, confirm these inputs. If any is unknown or vague, ASK — do not assume:
--latency-tolerance and the batch-vs-realtime verdict)--requests, --avg-input-tokens, --avg-output-tokens for the cost estimate)--realtime-input-price, --realtime-output-price, --batch-discount; defaults are neutral placeholders, not real prices)Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions.
cd engineering/batch-api-orchestrator
# 1. Should this be batch or realtime, and what does it cost?
python scripts/batch_cost_estimator.py \
--requests 50000 --avg-input-tokens 800 --avg-output-tokens 200 \
--realtime-input-price 3.0 --realtime-output-price 15.0 \
--batch-discount 0.5 --latency-tolerance hours
# 2. Plan the chunking / idempotency / retry strategy for the job
python scripts/batch_job_planner.py \
--total-items 50000 --max-batch-size 10000 --retry-policy exponential --json
| Tool | Purpose | Key Flags |
|---|---|---|
scripts/batch_cost_estimator.py | Compare realtime vs batch cost, show savings, and recommend batch or realtime given latency tolerance | --requests, --avg-input-tokens, --avg-output-tokens, --realtime-input-price, --realtime-output-price, --batch-discount, --latency-tolerance, --json |
test cases passed, vs 11 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.