Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mkurman/trl
Transformer Reinforcement Learning library (TRL). Supervised fine-tuning (SFT), reward modeling, PPO, DPO, KTO, GRPO for RLHF. Process reward models and language model alignment.
Passed 11 of 25 test cases with the skill, and 14 of 25 without it.
and 3 more in the full report
6 fixed10 passed either way5 still failing4 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
When instantiating AutoTokenizer.from_pretrained for this fine-tuning setup, what model identifier string is passed?
| Checked | Without skill | With skill |
|---|---|---|
| Passes AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct") | Missed | Passed |
The instructions your model reads when the skill loads.
TRL (Transformer Reinforcement Learning) is Hugging Face's library for RLHF — SFT, reward modeling, PPO, DPO, KTO, and GRPO. It's the standard post-training toolkit for aligning language models with human preferences.
uv pip install trl
from trl import SFTTrainer
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
trainer = SFTTrainer(
model=model, tokenizer=tokenizer,
train_dataset=dataset,
args=dict(per_device_train_batch_size=4, learning_rate=2e-5, max_seq_length=2048),
)
trainer.train()
from trl import DPOTrainer
dpo = DPOTrainer(
model=model, ref_model=ref_model, tokenizer=tokenizer,
train_dataset=preference_dataset,
args=dict(per_device_train_batch_size=4, max_length=2048),
)
dpo.train()
test cases passed, vs 14 / 25 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.