Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/tao-generate-video-reasoning-annotations
Multi-step video annotation pipeline that turns raw videos into Chain-of-Thought training data — multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with reasoning traces, via VLM/LLM distillation.
Passed 16 of 20 test cases with the skill, and 4 of 20 without it.
and 24 more in the full report
14 fixed4 passed either way4 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 12 test cases the skill fixed. Same model, same prompt, with and without the skill.
I want to generate a baseline YAML configuration file for the TAO auto-labeling tool. What command line invocation creates this default specification?
| Checked | Without skill | With skill |
|---|---|---|
| Uses auto_label default_specs command to generate baseline configuration | Missed | Passed |
The instructions your model reads when the skill loads.
Generate Chain-of-Thought training datasets from videos by producing multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with step-by-step reasoning traces. Domain-agnostic by default — customize prompts for any video domain.
Transform raw videos into CoT Q&A training data for video understanding models. VLMs (e.g., Gemini, Qwen) act as "teacher" annotators: Steps 0–1 require the model to see the video (VLM calls); Steps 2–3 are text-to-text (cheaper LLM calls).
Step 0: [Optional] Filter & classify videos → Keep domain-relevant, classify anomaly vs normal
Step 1a: Global + dense captions → VLM: narrative summary + timestamped events
Step 1b: Chunk captions → VLM: fixed-duration segment micro-captions
Step 1c: [Optional, anomaly only] Highlight → LLM extracts anomaly timestamp, VLM captions clip
Step 2: Description synthesis → LLM: synthesize captions into structured narrative
Step 3: QA generation → LLM: MCQ, binary, open-ended with reasoning
Step 4: Parse outputs → Per-task `tao-vl-reason-v1.0` JSON files
Steps are individually selectable via workflow.steps. The pipeline has built-in resume — each step skips already-processed videos, so re-running after a prompt tweak is safe.
When the user invokes this skill, walk through these questions in order. Don't skip — getting domain and VLM access right up front prevents wasted runs.
{"video_path": "..."} per line..mp4 preferred; .avi, .mov, .mkv also walked).Ask the user: "What domain are these videos from?" Choose one of the following branches:
| Domain | What to do |
|---|---|
| general | Use the default prompts. Set prompts_module: "" (or omit). The built-in nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts covers domain-agnostic content. |
| traffic (CCTV intersections, highways; dashcam excluded) | Use the reference module. Set prompts_module: "nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts_traffic", or copy references/prompts_traffic.py into the user's project and tune for their specific camera angles, then point prompts_module at the copy. |
| warehouse (industrial site CCTV — safety, operations, security) | Same pattern. Set prompts_module: "nvidia_tao_ds.auto_label.video_reasoning_annotation.prompts_warehouse", or copy references/prompts_warehouse.py and tune. |
test cases passed, vs 4 / 20 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.