Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →nvidia/tao-finetune-clip
CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX export, and TensorRT deployment.
Passed 19 of 20 test cases with the skill, and 8 of 20 without it.
and 18 more in the full report
12 fixed9 passed either way1 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
When AutoML is enabled for a TAO CLIP fine-tuning run, which two learning rate parameters are selected by default in the packaged AutoML schema for hyperparameter optimization?
| Checked | Without skill | With skill |
|---|---|---|
| Includes train.optim.vision_lr as an AutoML search parameter | Missed | Passed |
| Includes train.optim.text_lr as an AutoML search parameter | Missed | Passed |
The instructions your model reads when the skill loads.
Contrastive Language-Image Pre-training model for zero-shot and fine-tuned image classification, image-text retrieval, and embedding extraction. Fine-tuning adapts CLIP's shared image-text embedding space to domain-specific image-caption data.
No default NGC pretrained checkpoint is required for spec construction, but unset checkpoint behavior is action-specific. In the validation-fixes PyTorch image, export.checkpoint: null exports the selected CLIP architecture and may initialize weights when pretrained weights are unavailable. Do not assume inference.checkpoint: null loads pretrained weights: clip inference currently calls the checkpoint loader with None and fails before embedding extraction. For PyTorch inference, checkpoint-backed evaluation/export, resume, and retrain flows, resolve and pass an exact checkpoint from the parent train output. For trusted TAO checkpoints produced by the current run or a known parent job, set TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 on checkpoint-dependent PyTorch actions so PyTorch 2.6 can load the Lightning checkpoint metadata; do not set this for untrusted checkpoints.
Supported actions: train, evaluate, inference, export, gen_trt_engine.
This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.
The packaged CLIP train schema enables train.optim.vision_lr and train.optim.text_lr as default AutoML search parameters. For smoke tests, keep the search small by using the Bayesian algorithm with two recommendations and narrow LR ranges.
Non-train actions such as evaluate, inference, export, and deploy flows stay in this model skill. The per-run automl_policy override does not change model metadata.
Use this skill for NVIDIA TAO CLIP jobs: training, evaluation, embedding inference, ONNX export, and TensorRT engine generation. Start by identifying the requested action, then load only the referenced files needed for that action: defaults.json for default parameters, config.json for action/data-source wiring, references/spec_template.yaml for full spec shape, and references/model_info.yaml for SDK metadata.
For dataset-backed actions, collect the required image, caption, list, or prompt files from the user and place the resolved paths in spec_overrides. For local Docker runs, mount extracted folders in the container and point image_dir / caption_dir at those folders; if a data source provides .tar.gz archives, extract them before running the in-container CLIP commands. For export and gen_trt_engine, infer parent artifacts from the upstream job when available; otherwise require explicit checkpoint, ONNX, or engine paths. Run gen_trt_engine, TensorRT evaluate, and TensorRT inference in the TAO Deploy image.
For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, and TensorRT inference), read references/tao-deploy-clip.md first. Deploy spec templates live in this skill's references/ folder with the spec_template_deploy_*.yaml prefix.
ViT-L-14-SigLIP-CLIPA-224 (default), ViT-L-14-SigLIP-CLIPA-336, ViT-H-14-SigLIP-CLIPA-224, ViT-H-14-SigLIP-CLIPA-336, ViT-H-14-SigLIP-CLIPA-574c-radio_v3-b, c-radio_v3-l, c-radio_v3-h, c-radio_v3-gsiglip2-so400m-patch16-256, siglip2-so400m-patch14-224, siglip2-so400m-patch14-384, siglip2-so400m-patch16-384, siglip2-so400m-patch16-512, siglip2-so400m-patch16-naflexRadio-CLIP requires model.adaptor_name to be set to siglip or clip.
test cases passed, vs 8 / 20 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.