Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →waybarrios/hf-cloud-serving-image-selection
Pick the right serving container for a SageMaker model deployment and find its current image URI.
Passed 21 of 22 test cases with the skill, and 13 of 22 without it.
and 10 more in the full report
9 fixed12 passed either way0 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
When deploying `huggingface-vllm` with an image tag containing `cu130` on an `ml.g5.2xlarge` instance, what specific AMI parameter must be configured on the production variant?
| Checked | Without skill | With skill |
|---|---|---|
| specifies setting InferenceAmiVersion to al2-ami-sagemaker-inference-gpu-3-1 | Missed | Passed |
The instructions your model reads when the skill loads.
The serving container is the single thing most likely to break a SageMaker deployment that "looked correct on paper". Wrong container, stale tag, or the wrong AMI — all produce the same opaque Failed to pass health check error.
When both a HuggingFace-curated family (huggingface-vllm, huggingface-vllm-omni, huggingface-sglang, tei, huggingface-pytorch-inference) and a generic family (vllm, vllm-omni, sglang, djl-inference) can serve the model, the HuggingFace one is mandatory, not preferred. The only valid reasons to use a generic image:
A newer version number on the generic repo is not a reason. The AWS vllm repo often publishes a higher vLLM version than huggingface-vllm; an older-but-compatible huggingface-vllm tag still wins. "Latest vLLM" is not a requirement anyone stated — compatibility with the model is. If you fall back, record in the deployment log which of the three reasons applied.
Primary source: AWS's official Deep Learning Containers catalog.
URL: https://aws.github.io/deep-learning-containers/reference/available_images/
This page is AWS-maintained and lists every image family with example URIs, tags, CUDA versions, Python versions, and platform (SageMaker vs EC2/ECS/EKS). When picking a URI for a deployment, read it from this page directly — copy the example URL, substitute <region> with the user's region, and pass it to deploy.py --image-uri.
The example URLs use 763104351884 as the account ID for most regions. A few regions use different accounts (e.g. eu-south-1 uses 692866216735). Check the Region Availability page when in doubt.
Exception: none currently. Every image family used by this workflow is now on the AWS catalog page (TEI was added in late 2026). If you encounter a new family that isn't there, mirror it via mirror_image.py and pass the resulting URI directly.
| Model | Container family | How to get the URI |
|---|---|---|
| HuggingFace text-generation LLM (Llama, Qwen, Mistral, etc.) | HuggingFace vLLM | AWS catalog → "HuggingFace vLLM Inference" (ECR repo huggingface-vllm) |
| Same as above, multimodal | HuggingFace vLLM-Omni | AWS catalog → "HuggingFace vLLM-Omni Inference" (ECR repo huggingface-vllm-omni) |
| HuggingFace embeddings | TEI | AWS catalog → "HuggingFace Text Embeddings Inference" |
Encoder / cross-encoder rerankers (BERT-family *ForSequenceClassification) | TEI | Same as embeddings |
| Generative rerankers (causal-LM, e.g. Qwen3-Reranker) | HuggingFace vLLM | Same as text-generation LLMs — not TEI, see "Rerankers: TEI or vLLM?" |
| Text-to-image / diffusion (Stable Diffusion, FLUX) | DJL Inference | AWS catalog → "DJL Inference" — not HF Inference Toolkit, see "Known-broken images" |
| HuggingFace classifiers, NER, QA, summarization | HF Inference Toolkit (CPU) | AWS catalog → "HuggingFace PyTorch Inference"; GPU tags currently broken — see "Known-broken images" |
| User specifically wants SGLang | HuggingFace SGLang | AWS catalog → "HuggingFace SGLang Inference" |
No compatible huggingface-vllm tag (verified incompatibility or region gap — see "Rule zero") | vLLM (AWS) | AWS catalog → "vLLM" section — fallback only, never for version freshness |
| User specifically wants DJL-LMI | DJL Inference | AWS catalog → "DJL Inference" |
test cases passed, vs 13 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.