Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →wshobson/lora-qlora-recipes
Configure LoRA and QLoRA supervised fine-tuning with current best-practice hyperparameters.
Passed 21 of 23 test cases with the skill, and 9 of 23 without it.
and 11 more in the full report
12 fixed9 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 12 test cases the skill fixed. Same model, same prompt, with and without the skill.
To ensure bit-for-bit reproducibility of adapter matrix initialization across multiple experiment runs, which parameter should be explicitly set in the model adapter call?
| Checked | Without skill | With skill |
|---|---|---|
| The random_state parameter is explicitly defined in the adapter call. | Missed | Passed |
The instructions your model reads when the skill loads.
This skill assumes the routing decision already
happened — finetuning-method-selection should
have already pointed here because the data shape
is demonstrations (SFT), not preference pairs or
a verifiable reward signal. What follows is the
current best-practice recipe for configuring the
adapter itself: which modules to target, how to
size rank and alpha, what learning rate to use,
and when QLoRA buys real headroom versus when it
just adds risk. Dataset preparation and quality
checks are a separate concern — see
dataset-curation.
Input: a routing decision (SFT via LoRA/
QLoRA) plus a target size class.
Output format: a validated adapter config —
the kwarg values below, not free-form advice —
that llm-finetuning-training-engineer consumes
directly when it generates a runnable script.
The reference recipe is "LoRA Without Regret" (Thinking Machines/Schulman, 2025-09), now the settled convention for LoRA/QLoRA SFT.
Target all-linear modules, not just attention:
target_modules = [
"q_proj", "k_proj", "v_proj", "o_proj", # attention
"gate_proj", "up_proj", "down_proj", # MLP — matters most
]
test cases passed, vs 9 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.