TAO VCN Classify Gap Analysis Skill

TAO VCN Classify Gap Analysis Skill

You are an analyst for NVIDIA TAO VCN Classify (Visual Component Net) inference results. Your job is to identify the weakest samples per ground-truth label by measuring signed distance from the decision threshold in the wrong direction, then surface them for downstream augmentation or relabeling.

This skill is intentionally lightweight. VCN's classify head is a single-score binary boundary (PASS vs NO_PASS by siamese_score), so the analysis is computational, not investigative. The whole computation lives behind one direct docker run invocation against the tao_toolkit.data_services image declared in versions.yaml (resolved at runtime — see Setup). The container's entrypoint takes <category> <action> [hydra overrides...]; we pass gap_analysis vcn_aoi key=value …. Each override is a bare Hydra key=value that selectively overrides the script's GapAnalysisConfig schema (defaults are baked into the container; introspect with docker run ... gap_analysis vcn_aoi --cfg=job). (There is no dataset keyword inside the container — that's the TAO launcher's pillar prefix and is dropped here.) You do not need delegated analysis, multi-phase image audits, or component-type clustering — VCN does not expose those dimensions. View only a small set of representative weak samples to qualify the gaps after the container returns.

CLI surface can shift between data-services container builds. If a gap_analysis vcn_aoi invocation fails on argument parsing, introspect the actual schema once per image with docker run --rm "$DS_IMAGE" gap_analysis vcn_aoi --cfg=job and reconcile any renamed keys (e.g. inference_csv vs inference_results_dir, output_dir vs results_dir) before retrying. Output parquet name is kpi_gaps.parquet.

Inputs

1. Experiment result directory — contains inference/inference.csv from TAO VCN Classify inference. Required columns: input_path, object_name, label, siamese_score. Pass the directory (e.g. inference/latest/), not the CSV file — the container reads inference_results_dir/inference.csv.
2. Training code/config directory — contains the VCN train YAML. The container reads dataset.classify.input_map (lighting condition list) and dataset.classify.image_ext from it to expand each weak sample into one row per lighting.
3. Dataset directory — image root prepended to the relative input_path from each row (kpi_media_path).
4. Schema overrides — min_recall, top_k_per_label, and optionally a hard-pinned threshold are passed as Hydra overrides (defaults: min_recall=1.0, top_k_per_label=50, threshold=-1.0 meaning sweep). top_k_per_label must be a positive integer — omitting it flips the container into "below-threshold filter" mode, which at min_recall=1.0 returns only PASS misclassifications and zero NO_PASS rows. See Common pitfalls.

Setup

The threshold sweep, weakness ranking, and per-lighting expansion all run inside the tao_toolkit.data_services image declared in versions.yaml. Resolve the concrete URI once at the top of the run, then confirm Docker, the NVIDIA container toolkit, and a GPU are present and ensure the image is cached:

# Resolve tao_toolkit.data_services → concrete nvcr.io/... URI from versions.yaml
DS_IMAGE=$(python3 -c "import yaml,os; print(yaml.safe_load(open(os.environ['TAO_SKILL_BANK_PATH']+'/versions.yaml'))['images']['tao_toolkit']['data_services'])")
echo "DS_IMAGE=$DS_IMAGE"

docker info > /dev/null && echo "OK: docker"
nvidia-smi > /dev/null && echo "OK: GPU"
docker image inspect "$DS_IMAGE" > /dev/null \
  || docker pull "$DS_IMAGE"

TAO_SKILL_BANK_PATH is usually exported by the installed skill bank. If it is unset, point it at the skill-bank repo root before resolving. A GPU is required; aborting early on a GPU-less host saves a confusing late error.

Three setup rules are load-bearing and easy to get wrong:

• Path mounting — every host path the container reads or writes (inference.csv, train YAML, dataset image root, output dir) must be bind-mounted, simplest with -v $WORKSPACE:$WORKSPACE -w $WORKSPACE so absolute paths resolve identically on both sides.
• Do not pass --user $(id -u):$(id -g) — it triggers KeyError: 'getpwuid(): uid not found: <uid>' during the container's transformers import; chown outputs back to the host UID afterwards instead.