Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →didhd/sagemaker-deploy
Deploy an open-weight LLM to a SageMaker AI real-time endpoint as an Inference Component, using the latest vLLM Deep Learning Container, a GPU instance sized to the model, tensor-parallel set to the GPU count, and model weights staged in S3.
Passed 16 of 19 test cases with the skill, and 8 of 19 without it.
and 9 more in the full report
10 fixed8 passed either way3 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
The instructions your model reads when the skill loads.
The portable contract that makes a non-deterministic agent definitive. Agents improvise; this SKILL.md constrains what must hold (the APIs, the ordering, the compatibility rules) while leaving how open as explicit, bounded options. The IDE is interchangeable — this contract is what travels.
Deploy a HuggingFace SafeTensor OSS model from S3 to a SageMaker AI real-time
endpoint via an Inference Component (IC). Plain deploy — measurement is the
sagemaker-benchmark skill, and optimization is the sagemaker-optimize skill. Path is fixed:
CreateModel → CreateEndpointConfig → CreateEndpoint → CreateInferenceComponent → smoke test.
This skill is model-agnostic. Deploy whatever open-weight model the user names; you only need three facts about it, which then drive Decisions 2–4:
| Property | Why it matters | Drives |
|---|---|---|
| Weights size (GB) + dtype | must fit in GPU memory alongside the KV cache | instance choice (Decision 2) |
| vLLM support | the DLC serves models with a native vLLM implementation | instance + go/no-go |
| Context length needed | longer context = more KV-cache memory | max_model_len (Decision 4) |
Default running example: GPT-OSS-20B (openai/gpt-oss-20b, ~13 GB mxfp4, text +
reasoning) — a strong open model that is not one-click in JumpStart, i.e. the exact
"raw weights → production" case this skill is for. If the user names another model, deploy
that one; if unspecified, ask. The same path works for Llama, Mistral, Qwen, etc.
Rule: pick the smallest instance whose GPU memory comfortably holds weights + KV cache,
then set tensor_parallel_size = GPU count.
| Instance | GPU | GPU mem | Good for |
|---|---|---|---|
ml.g6.16xlarge | 1× L4 | 24 GB | ≤13B, or ~20B quantized (GPT-OSS-20B mxfp4 fits); reliable capacity |
ml.g6.24xlarge | 4× L4 | 96 GB | larger / TP=4; recommendation-job target |
ml.g6e.16xlarge | 1× L40S | 48 GB | single big GPU — ~30B or long context |
test cases passed, vs 8 / 19 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.