Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Trl · mkurman | DecimalAI

mkurman/trl

NewSkillScoreMaintained

Transformer Reinforcement Learning library (TRL). Supervised fine-tuning (SFT), reward modeling, PPO, DPO, KTO, GRPO for RLHF. Process reward models and language model alignment.

AI/MLby mkurman

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

AI Engineer pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/mkurman-trl/SKILL.md

decimalai skills pull mkurman-trl --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/mkurman-trl/SKILL.md https://app.decimal.ai/s/mkurman-trl/SKILL.md

Raw file: /s/mkurman-trl/SKILL.md ↗ · machine-readable summary at /s/mkurman-trl.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

25 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 8d ago

No — -11% lift-8% tokens0% turns64% of whole cases pass with skill (16/25)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-07	✗→✓	▲ Improved	-47%	0%
case-09	✗→✓	▲ Improved	-15%	0%
case-15	✗→✓	▲ Improved	-7%	0%
case-19	✗→✓	▲ Improved	-19%	0%
case-23	✗→✓	▲ Improved	63%	0%

Showing 5 of 25 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

mkurman / zorai

skills/scientific-skills/trl/SKILL.md

View on GitHub ↗

License MITImported 8/3/2026Version v1

Skill contents

SKILL.md

Overview

TRL (Transformer Reinforcement Learning) is Hugging Face's library for RLHF — SFT, reward modeling, PPO, DPO, KTO, and GRPO. It's the standard post-training toolkit for aligning language models with human preferences.

Installation

bash
uv pip install trl

SFT

python
from trl import SFTTrainer
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")

trainer = SFTTrainer(
    model=model, tokenizer=tokenizer,
    train_dataset=dataset,
    args=dict(per_device_train_batch_size=4, learning_rate=2e-5, max_seq_length=2048),
)
trainer.train()

DPO

python
from trl import DPOTrainer

dpo = DPOTrainer(
    model=model, ref_model=ref_model, tokenizer=tokenizer,
    train_dataset=preference_dataset,
    args=dict(per_device_train_batch_size=4, max_length=2048),
)
dpo.train()

References

Want to see if mkurman/trl would activate on your phrasing? Try it before you install — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/21/2026, 17:34 UTC · 25 cases

Lift

-11%

44% vs 56%

Agent turns

1.0 vs 1.0

Tokens

-8%

1,208.48 vs 1,314.48

fail → pass · 6pass → pass · 10fail → fail · 5pass → fail · 4

Per-case results (25)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 I need a Python script to perform supervised fine-tuning on a causal language model using Hugging Face's TRL library. Could you show me how to initialize and execute the SFT trainer with a custom dataset and training parameters?	fail→fail	17,234	11,570	-33%	1	1	0%	3,397	2,696	-21%	0	0	—
case-02 How do I install the TRL package via uv, and what is the code snippet to quickly set up SFTTrainer with a pre-trained tokenizer and model?	fail→fail	11,215	5,795	-48%	1	1	0%	2,127	1,558	-27%	0	0	—
case-03 How do I fine-tune a model using PyTorch's native torch.nn.Module and raw training loop with AdamW optimizer without using Hugging Face High-Level Trainer abstractions?	pass→pass	17,931	17,260	-4%	1	1	0%	3,553	4,043	+14%	0	0	—
case-04 How do I configure a vLLM inference server using Docker or python CLI to host a Qwen model for high-throughput generation?	pass→pass	15,788	14,647	-7%	1	1	0%	3,027	3,287	+9%	0	0	—
case-05 How do I implement standard parameter-efficient fine-tuning (PEFT) with LoRA on a base model using Hugging Face peft library's get_peft_model and LoraConfig directly?	pass→pass	13,972	9,352	-33%	1	1	0%	2,773	2,184	-21%	0	0	—
case-06 Provide the exact terminal command required to install the TRL library into a virtual environment using the uv package manager.	pass→pass	3,269	1,736	-47%	1	1	0%	491	598	+22%	0	0	—
case-07 When loading a model for supervised fine-tuning with AutoModelForCausalLM in the standard workflow, which Qwen checkpoint identifier is specified?	fail→pass	6,934	3,092	-55%	1	1	0%	1,254	662	-47%	0	0	—
case-08 When configuring batch size options in the args dictionary for SFTTrainer, which key and integer value are used for per-device training batch size?	pass→pass	4,290	2,767	-36%	1	1	0%	705	773	+10%	0	0	—
case-09 In SFTTrainer, how is the sequence length limit configured inside the args parameter dictionary?	fail→pass	6,542	3,921	-40%	1	1	0%	1,203	1,027	-15%	0	0	—
case-10 What learning rate parameter value is supplied inside the args dictionary during SFTTrainer setup?	fail→fail	7,085	1,769	-75%	1	1	0%	1,302	616	-53%	0	0	—
case-11 Write a python import statement to import the supervised fine-tuning class from the trl module.	pass→pass	1,869	1,353	-28%	1	1	0%	247	505	+104%	0	0	—
case-12 Once the SFTTrainer instance named trainer has been initialized, what method call begins the model training process?	pass→pass	2,102	1,544	-27%	1	1	0%	270	566	+110%	0	0	—
case-13 Write the python import statement required to bring Direct Preference Optimization support from the trl package.	pass→pass	1,998	2,038	+2%	1	1	0%	300	616	+105%	0	0	—
case-14 When initializing DPOTrainer, what model arguments are provided to pass both the policy model and the baseline reference model?	pass→fail	5,801	4,993	-14%	1	1	0%	996	1,226	+23%	0	0	—
case-15 What dataset keyword argument name is passed into DPOTrainer to provide the preference alignment data?	fail→pass	4,646	2,397	-48%	1	1	0%	778	726	-7%	0	0	—
case-16 In DPOTrainer, what key name and integer value inside args=dict(...) specify the maximum token length limit?	fail→fail	4,893	2,527	-48%	1	1	0%	897	744	-17%	0	0	—
case-17 After creating a DPOTrainer object assigned to variable dpo, how is training executed?	pass→pass	4,364	1,578	-64%	1	1	0%	735	550	-25%	0	0	—
case-18 Which keyword argument name is used when passing the training dataset to SFTTrainer?	pass→fail	2,362	1,486	-37%	1	1	0%	367	580	+58%	0	0	—
case-19 How are hyperparameter options like batch size and learning rate passed directly into SFTTrainer without using a separate config object?	fail→pass	14,053	9,582	-32%	1	1	0%	2,518	2,027	-19%	0	0	—
case-20 What keyword argument is passed to SFTTrainer to supply the loaded tokenizer object?	pass→fail	5,150	2,987	-42%	1	1	0%	824	840	+2%	0	0	—
case-21 How is the tokenizer object provided to DPOTrainer during instantiation?	fail→fail	9,573	8,121	-15%	1	1	0%	1,521	1,655	+9%	0	0	—
case-22 What per-device batch size integer is passed inside args=dict(...) for DPOTrainer?	pass→fail	4,938	1,926	-61%	1	1	0%	932	674	-28%	0	0	—
case-23 When instantiating AutoTokenizer.from_pretrained for this fine-tuning setup, what model identifier string is passed?	fail→pass	2,642	2,195	-17%	1	1	0%	401	653	+63%	0	0	—
case-24 When initializing AutoModelForCausalLM.from_pretrained, which model repository string is provided?	fail→pass	6,415	1,804	-72%	1	1	0%	1,127	610	-46%	0	0	—
case-25 What parameter keyword difference exists for configuring maximum context length between SFTTrainer args vs DPOTrainer args?	pass→pass	6,418	2,357	-63%	1	1	0%	1,117	796	-29%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 25 cases were attempted. The headline lift of -11 percentage points is the difference between those two pass rates over the 25 comparable cases. 5 cases got worse with the skill loaded, and they are included in that figure.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 55.6%
Pass rate with the skill: 44.4%
Cases attempted: 25
Cases counted toward lift: 25
Cases that regressed: 5
Skill version measured: v1
Run completed: 8/21/2026, 17:34 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-07

Prompt

When loading a model for supervised fine-tuning with AutoModelForCausalLM in the standard workflow, which Qwen checkpoint identifier is specified?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signalsno signals yet

BenchmarkRun a verified benchmark with positive lift

Live evalNeeds ≥20 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+5% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

No signals yet — run a benchmark or collect evaluated traces to earn a score.

Verified on gemini-3.6-flash · re-tested 7 days ago

Other measured skills in the registry, with their headline benchmark lift.