Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →didhd/sagemaker-optimize
Find an optimized serving configuration for a model with Amazon SageMaker AI inference recommendations, then deploy it — so you can show a before/after speed-up versus the baseline benchmark. Covers both config search (best instance + serving knobs) and deep optimization (speculative decoding / EAGLE 3, quantization, kernel tuning).
Passed 18 of 20 test cases with the skill, and 5 of 20 without it.
and 14 more in the full report
14 fixed6 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
The instructions your model reads when the skill loads.
The "make it faster" contract. A plain deploy gets the model serving; this gets it serving well — and proves it with a number. SageMaker AI inference recommendations searches serving configurations on managed compute and hands back the best one it found, including the expected performance. You never hand-tune vLLM flags or build a sweep harness. This is what a one-click JumpStart deploy does not give you.
Pairs with
sagemaker-benchmark: benchmark the baseline → optimize → benchmark again → show before/after throughput. That delta is the headline.
create_ai_recommendation_job)| Depth | OptimizeModel | What you get | Cost / time | Live on stage? |
|---|---|---|---|---|
| Config search | False | Best instance + serving config for the model as-is; ExpectedPerformance | minutes, mid-size instance (e.g. ml.g6.24xlarge) | yes (or pre-bake) |
| Deep optimize | True | Above + speculative decoding (EAGLE 3), quantization, kernel tuning, registered as a Model Package | hours, large instance (e.g. ml.p5en.48xlarge), usually capacity-reserved | no — pre-bake it |
Stage guidance: the deep-optimize path is slow and needs scarce reserved capacity, so pre-bake its result (run it the day before) and show the saved before/after on stage — like a cooking show pulling the finished dish from the oven. Don't run a multi-hour job live.
create_ai_workload_config → create_ai_recommendation_job →
poll describe_ai_recommendation_job → deploy the result's Model Package
(create_model from ModelPackageName → create_endpoint_config → create_endpoint).
create_ai_workload_config describing your traffic (sharegpt, or a
custom JSONL dataset in S3 via DatasetConfig.InputDataConfig with a folder S3Uri).
Dataset-format asymmetry vs the benchmark service: the recommendation service
validates the dataset files and accepts only ShareGPT or OpenAI Chat/Completions
records — it rejects the benchmark's AIPerf single_turn lines ({"text", ...}) with
"unrecognized format". Use the bundled datasets/sharegpt-curated-openai.jsonl (the
benchmark skill's curated prompts converted to OpenAI Chat format) so the
recommendation optimizes for the same traffic the benchmark measures.create_ai_recommendation_job with ModelSource.S3,
PerformanceTarget.Constraints=[{Metric: throughput|latency}],
ComputeSpec.InstanceTypes=[...], InferenceSpecification.Framework (VLLM for config
search, LMI for deep optimize), and OptimizeModel (False/True). For scarce instancestest cases passed, vs 5 / 20 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.