Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →aws-samples/sagemaker-optimize
Find an optimized serving configuration for a model with Amazon SageMaker AI inference recommendations, then deploy it and compare against a baseline benchmark. Covers both config search (best instance + serving knobs) and deep optimization (speculative decoding / EAGLE 3, quantization, kernel tuning).
Passed 19 of 22 test cases with the skill, and 6 of 22 without it.
and 19 more in the full report
15 fixed5 passed either way2 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 15 test cases the skill fixed. Same model, same prompt, with and without the skill.
We want to test our custom OpenAI-formatted JSONL prompt file with scripts/recommend.py. Which CLI argument stages our local dataset file to S3 automatically for the recommendation job?
| Checked | Without skill | With skill |
|---|---|---|
| The response passes the --dataset-file argument to scripts/recommend.py. | Missed | Passed |
The instructions your model reads when the skill loads.
SageMaker AI inference recommendations searches serving configurations on managed compute and hands back the best one it found, including the expected performance. You never hand-tune vLLM flags or build a sweep harness.
Pairs with
sagemaker-benchmark: benchmark the baseline → optimize → benchmark again → compare like-for-like before/after throughput.
create_ai_recommendation_job)| Depth | OptimizeModel | What you get | Cost / time | Execution |
|---|---|---|---|---|
| Config search | False | Best instance + serving config for the model as-is; ExpectedPerformance | minutes to hours, workload-dependent | interactive or automated |
| Deep optimize | True | Configuration plus supported model optimizations, registered as a Model Package | hours, large capacity-reserved instance | run asynchronously |
Deep optimization is slow and commonly needs scarce reserved capacity. Run it asynchronously and retain its result rather than coupling it to an interactive workflow.
create_ai_workload_config → create_ai_recommendation_job →
poll describe_ai_recommendation_job → deploy the result's Model Package
(create_model from ModelPackageName → create_endpoint_config → create_endpoint).
create_ai_workload_config describing your traffic (sharegpt, or a
custom JSONL dataset in S3 via DatasetConfig.InputDataConfig with a folder S3Uri).
Dataset-format asymmetry vs the benchmark service: the recommendation service
validates the dataset files and accepts only ShareGPT or OpenAI Chat/Completions
records — it rejects the benchmark's AIPerf single_turn lines ({"text", ...}) with
"unrecognized format". Use the bundled datasets/sharegpt-curated-openai.jsonl (the
benchmark skill's curated prompts converted to OpenAI Chat format) so the
recommendation optimizes for the same traffic the benchmark measures.create_ai_recommendation_job with ModelSource.S3,
PerformanceTarget.Constraints=[{Metric: throughput|latency}],
ComputeSpec.InstanceTypes=[...], InferenceSpecification.Framework (VLLM for config
search, LMI for deep optimize), and OptimizeModel (False/True). For scarce instances
add ComputeSpec.CapacityReservationConfig (capacity-reservations-only +test cases passed, vs 6 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.