See it in action
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
PromptWe want to use RLAIF with a custom judge prompt for our reward model fine-tuning step. Which template file should be loaded?
What was checked, without and with the skill| Checked | Without skill | With skill |
|---|
| References code_templates/rlaif_custom_prompt.py for custom prompt RLAIF | Missed | Passed |
|---|
What's inside
The instructions your model reads when the skill loads.
SKILL.md175 lines9 filesincludes scripts
Prerequisites
Before starting this workflow, verify:
-
A use_case_spec.md file exists
- If missing: Activate the
use-case-specification skill first, then resume
- DON'T EVER offer to create a use case spec without activating the use-case-specification skill.
-
A fine-tuning technique (SFT, DPO, RLVR, RLAIF, or CPT/RFT (for Nova)) and base model have already been selected
- If missing: Activate the
model-selection and/or finetuning-technique skills to collect what's missing, then resume
- Don't make recommendations on the spot. You MUST activate the appropriate skill.
-
A base model name available on SageMakerHub has been identified
- If missing: Activate the
model-selection skill to get it
- Important: Only use the model name that
model-selection retrieves, as it may differ from other commonly used names for the same model
-
The SDK environment has been verified (SDK version, region, execution role)
- If not done: Activate the
sdk-getting-started skill first, then resume
-
A training dataset uploaded to a bucket in the environment's default region.
- If not met: Help the user upload the dataset to the correct S3
Critical Rules
Code Generation Rules
- ✅ Use EXACTLY the imports shown in each code template
- ❌ Do NOT add additional imports even if they seem helpful
- ❌ Do NOT create variables before they're needed in that section
- 📋 Copy the code structure precisely - no improvisation
- 🎯 Follow the minimal code principle strictly
- ✅ When writing code, make sure the indentation and f strings are correct
User Communication Rules
- ❌ NEVER offer to move on to a downstream skill while training is in progress (logically impossible)
- ❌ NEVER set ACCEPT_EULA to True without explicit user confirmation in the conversation
- ✅ Always mention both the number AND title of sections you reference
Track record
Benchmark
15 / 23
test cases passed, vs 8 / 23 without
Adoption
- Users
- 0
- Times used
- 0
Safety and source details
SkillSafety
✓ 3 of 3 passed✓Code scan15 of 15 checks clean
0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026
View 15 scanner checks
live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing
✓IntentSafe
LLM judge · does the skill do what it claims
✓ContentClean
LLM judge · policy & harmful-content review
Source
Importedawslabs / agent-plugins
plugins/sagemaker-ai/skills/finetuning/SKILL.md
View on GitHub ↗License Apache-2.0Imported 8/3/2026Version v1Bundled 8 files · scripts