Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/tao-run-inference-service
Start, query, and stop a network-specific TAO inference microservice ({network_arch}-inference-microservice) by delegating container execution to the appropriate platform skill. Handles container image resolution, job-payload JSON construction, and the service registry.
Passed 15 of 18 test cases with the skill, and 7 of 18 without it.
and 24 more in the full report
11 fixed7 passed either way3 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.
I want to deploy a TAO inference microservice for my cosmos-predict2.5 checkpoint located at `hf_model://nvidia/cosmos-pred-2.5` on my local Docker setup using 1 GPU. Please walk me through the setup, generate the necessary startup script to launch the container service, write the local service state mapping, and include a loop that waits for the service to report readiness. Output the complete runnable Python script along with brief execution instructions.
| Checked | Without skill | With skill |
|---|---|---|
| The script generates a fresh UUID using uuid.uuid4() for the job_id. | Missed | Passed |
| The script constructs the container command starting with umask 0 && cosmos_predict inference_microservice start. | Missed | Passed |
| The script writes the service registry mapping to /tmp/tao-inf-ms-state.json. | Missed | Passed |
| The script maps host port to container port 8080. | Missed | Passed |
The instructions your model reads when the skill loads.
To start an inference service:
references/code-templates.yaml → job_payload_builder.skills/platform/<platform>/SKILL.md and start the container (Section 4.2).references/code-templates.yaml → registry_write.<platform> and readiness_check.To send an inference request:
job_id, by network_arch, or by explicit user choice when multiple services run — never silently default to "latest" when more than one service exists), then read the endpoint from references/code-templates.yaml → request.registry_read with the resolved job_id.max_tokens, top_p, temperature (and any per-arch extras) with their defaults; let the user override or skip each one to accept the default. Never silently use defaults.To stop a service: Read references/code-templates.yaml → stop.registry_read to resolve the job_id, read skills/platform/<platform>/SKILL.md, then follow Section 5.
Reference data (schemas, mappings, valid values — no instructions):
references/service.yaml — image mappings, valid network_arch names, job payload schema, env var names, secrets classification.references/request.yaml — endpoint definition, request field schema, response shapes, code examples.references/code-templates.yaml — Python templates for payload building, registry writes, readiness checks, and stop/request flows.Never ask the user to type a secret value into a prompt. For every secret value:
export HF_TOKEN=...).os.environ["VAR_NAME"] — never hard-code, interpolate, or prompt for the value.Secret env vars (full list in references/service.yaml → secrets_handling):
HF_TOKEN, WANDB_API_KEY, CLEARML_API_ACCESS_KEY, CLEARML_API_SECRET_KEY, TAO_API_KEY, TAO_USER_KEY.
Safe to collect in the prompt: network_arch, model_path, num_gpus, prompt text, WANDB_* config URLs, CLEARML_*_HOST URLs.
| Input | Role |
test cases passed, vs 7 / 18 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.