GRPO & RLVR Training

GRPO & RLVR Training

This skill assumes finetuning-method-selection already routed here because the target behavior has a verifiable pass/fail signal — not demonstrations (lora-qlora-recipes) or preference pairs (preference-optimization). What follows is when RL is the right tool, the reference recipe, the mandatory reward-inspection gate, and how to pick a GRPO variant when the base recipe misbehaves.

Input: a routing decision (RLVR via GRPO) plus a verifier (code executor, test suite, schema checker, or grader) for the target task. Output format: a validated GRPO config — the kwarg values in references/grpo-memory.md and the reward functions in references/reward-functions.md, not free-form advice — that llm-finetuning-training-engineer consumes directly.

When RL Applies

GRPO+RLVR only pays off when task success is algorithmically checkable — a unit test passes, a parser accepts the output, a tool call matches an expected schema, a math answer matches a ground truth. If grading the output requires human judgment or a subjective rubric, that's an eval-harness and judge-calibration problem first — see eval-harness-first — not a reason to skip straight to RL.

Before opening a GRPO run, confirm the model can sometimes succeed on the target task already. RL sharpens an existing capability by reweighting toward the samples that already work; it does not install a capability from zero.