Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →graniet/trl-fine-tuning
Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training.
Passed 16 of 22 test cases with the skill, and 15 of 22 without it.
1 fixed16 passed either way6 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
A test case the skill fixed. Same model, same prompt, with and without the skill.
I am estimating GPU VRAM requirements for aligning 7B parameter models. What are the minimum VRAM requirements for 7B DPO training versus 7B PPO training?
| Checked | Without skill | With skill |
|---|---|---|
| The response states 7B DPO requires 24GB VRAM. | Missed | Passed |
| The response states 7B PPO requires 40GB VRAM. | Passed | Passed |
The instructions your model reads when the skill loads.
This skill is repo-local and stays inactive until explicitly activated.
When the original instructions refer to legacy tool names, use these Kheish mappings:
terminal => bashweb_extract => web_fetch, plus web_search when discovery is neededsearch_files => grep_search and glob_searchbrowser_* tools require a browser-capable surfaced tool or MCP; if none is available, use the closest available surface and say so explicitlyWhen the instructions mention local helper files, resolve them from ${KHEISH_SKILL_DIR}.
TRL provides post-training methods for aligning language models with human preferences.
Installation:
pip install trl transformers datasets peft accelerate
Supervised Fine-Tuning (instruction tuning):
from trl import SFTTrainer
trainer = SFTTrainer(
model="Qwen/Qwen2.5-0.5B",
train_dataset=dataset, # Prompt-completion pairs
)
trainer.train()
DPO (align with preferences):
from trl import DPOTrainer, DPOConfig
config = DPOConfig(output_dir="model-dpo", beta=0.1)
test cases passed, vs 15 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.