Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Prepare, launch, and evaluate Kimi LoRA fine-tuning for Smithers TypeScript/JSX workflow authoring. Use when building a Smithers training corpus, validating Fireworks JSONL, comparing base and tuned Kimi models, starting a Fireworks Kimi K2.7 managed SFT job, requesting or using Kimi K3 Serverless Training access, estimating training spend, or refreshing the checked-in Smithers authoring dataset.
.claude/skills/smithersai-train-kimi-smithers/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-16 | ✗→✓ | ▲ Improved | -23% | 0% |
| case-01 | ✗→✓ | ▲ Improved | -18% | 0% |
| case-02 | ✗→✓ | ▲ Improved | 70% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 8% | 0% |
| case-05 | ✗→✓ | ▲ Improved | -17% | 0% |
Build the corpus locally, establish a base-model score, run the smallest useful LoRA SFT, and promote it only when the held-out benchmark improves.
Run from the Smithers repository root:
bashnode skills/train-kimi-smithers/scripts/prepare-dataset.mjs \ --out artifacts/kimi-smithers node skills/train-kimi-smithers/scripts/validate-dataset.mjs \ artifacts/kimi-smithers/train.jsonl \ artifacts/kimi-smithers/validation.jsonl
The builder uses only canonical workflow files listed in assets/source-manifest.json. It splits by source file, never by generated prompt variant, so validation cannot contain another wording of a training completion. Read methodology.md before changing sources, variants, or the system prompt.
Fireworks model is publicly marked tunable and coding-focused.
preview on the account. K3 is open-weight, but its public model card currently says fine-tuning is unavailable outside the preview.
only if the live K2.7 model or training shape is temporarily unavailable.
Read fireworks.md completely before account setup, upload, job creation, promotion, or deployment. Recheck every linked live model, price, and CLI page because provider support changes quickly.
After the account has an API key, score the public base model before training:
bashFIREWORKS_API_KEY=... node \ skills/train-kimi-smithers/scripts/evaluate-fireworks.mjs \ --model accounts/fireworks/models/kimi-k2p7-code \ --out artifacts/kimi-smithers/eval-base.json
The evaluator uses held-out prompts, checks required/forbidden authoring patterns, and renders each generated workflow through the real local smithers graph command. Treat a graph failure as a failure even when lexical checks pass.
For K2.7, use the exact managed SFT flow in fireworks.md. Keep the first run to one epoch, LoRA rank 8, a 32K maximum context, and the separate validation dataset. Leave the learning rate at the platform default.
For K3 preview, inspect the resolved configuration first:
bashpython skills/train-kimi-smithers/scripts/train_kimi_k3.py \ --dataset artifacts/kimi-smithers/train.jsonl \ --output-model-id smithers-kimi-k3-v1 \ --print-config
Run the same command with --confirm-spend only after preview access, live pricing, billing, and the displayed configuration are confirmed.
Deploy the completed LoRA temporarily, then rerun evaluate-fireworks.mjs with its full model resource name. Keep the tune only if:
Tear down the validation deployment after the comparison. Add real, consented request-to-workflow pairs and hard failures to the next corpus; never train on held-out cases or generated benchmark answers.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-16 | fail→pass | 9,054 | 1,771 | -80% | 1 | 1 | 0% | 1,464 | 1,123 | -23% | 0 | 0 | — |
case-01 | fail→pass | 17,856 | 10,986 | -38% | 1 | 1 | 0% | 3,801 | 3,116 | -18% | 0 | 0 | — |
case-02 | fail→pass | 8,374 | 8,394 | +0% | 1 | 1 | 0% | 1,443 | 2,447 | +70% | 0 | 0 | — |
case-03 | fail→pass | 14,142 | 10,088 | -29% | 1 | 1 | 0% | 2,733 | 2,957 | +8% | 0 | 0 | — |
case-04 | pass→pass | 12,084 | 7,674 | -36% | 1 | 1 | 0% | 2,187 | 2,153 | -2% | 0 | 0 | — |
case-05 | fail→pass | 13,718 | 5,023 | -63% | 1 | 1 | 0% | 2,322 | 1,924 | -17% | 0 | 0 | — |
case-06 | fail→pass | 36,107 | 3,257 | -91% | 1 | 1 | 0% | 1,541 | 1,462 | -5% | 0 | 0 | — |
case-07 | pass→pass | 8,949 | 2,861 | -68% | 1 | 1 | 0% | 1,417 | 1,271 | -10% | 0 | 0 | — |
case-08 | fail→pass | 11,958 | 4,343 | -64% | 1 | 1 | 0% | 2,280 | 1,624 | -29% | 0 | 0 | — |
case-09 | fail→pass | 12,549 | 1,788 | -86% | 1 | 1 | 0% | 2,351 | 1,152 | -51% | 0 | 0 | — |
case-10 | fail→pass | 11,197 | 2,732 | -76% | 1 | 1 | 0% | 1,706 | 1,383 | -19% | 0 | 0 | — |
case-15 | fail→pass | 11,889 | 4,769 | -60% | 1 | 1 | 0% | 1,868 | 1,606 | -14% | 0 | 0 | — |
case-11 | fail→pass | 10,741 | 2,102 | -80% | 1 | 1 | 0% | 1,865 | 1,238 | -34% | 0 | 0 | — |
case-12 | fail→pass | 11,395 | 4,409 | -61% | 1 | 1 | 0% | 1,851 | 1,701 | -8% | 0 | 0 | — |
case-13 | fail→pass | 13,932 | 5,008 | -64% | 1 | 1 | 0% | 2,265 | 1,690 | -25% | 0 | 0 | — |
case-14 | pass→pass | 9,087 | 4,015 | -56% | 1 | 1 | 0% | 1,488 | 1,435 | -4% | 0 | 0 | — |
case-17 | fail→pass | 11,982 | 1,890 | -84% | 1 | 1 | 0% | 379 | 1,138 | +200% | 0 | 0 | — |
case-18 | fail→pass | 6,381 | 2,364 | -63% | 1 | 1 | 0% | 1,096 | 1,318 | +20% | 0 | 0 | — |
case-19 | fail→pass | 8,405 | 2,007 | -76% | 1 | 1 | 0% | 1,434 | 1,195 | -17% | 0 | 0 | — |
case-20 | fail→pass | 8,137 | 2,643 | -68% | 1 | 1 | 0% | 1,467 | 1,260 | -14% | 0 | 0 | — |
case-21 | fail→fail | 7,064 | 2,190 | -69% | 1 | 1 | 0% | 1,250 | 1,197 | -4% | 0 | 0 | — |
case-22 | pass→pass | 3,199 | 3,710 | +16% | 1 | 1 | 0% | 526 | 1,424 | +171% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 20 counted toward the lift figure. The other 2 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +77 percentage points is the difference between those two pass rates over the 20 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.