Prepare and validate Parameter Golf record folders: self-contained train_gpt.py, README.md, submission.json, FineWeb SP1024 BPB accounting, artifact-size logging, run logs, and PR-ready folder hygiene.
SkillScore100Maintained
Automatic
◆AI Engineer pack
Last verified 8/21/2026
What it changes
Passed 23 of 23 test cases with the skill, and 9 of 23 without it.
Without skill39%
With skill100%
Checks it passed that the model alone missed
Assigns the status tag smoke_passed.
Includes val_bpb in submission.json.
Includes run_name in submission.json.
and 21 more in the full report
14 fixed9 passed either way0 still failing0 broke
Pass-rate lift
+61 pp
Tokens
+15%
Agent turns
0%
Response time
-43%
Model gemini-3.6-flashMeasured 8/21/2026v123 of 23 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 14 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 14
Prompt
I just created my submission folder and populated train_gpt.py and README.md, but I haven't executed the training run yet. What status value should I set in submission.json?
What was checked, without and with the skill
Checked
Without skill
With skill
Assigns the exact status string prepared_pending_run.
Missed
Passed
case-06 · gemini-3.6-flash · 8/21/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md107 linessingle file
Parameter Golf Submission
Use this skill when creating or reviewing a Parameter Golf submission folder, independent of the cloud provider used for the run.
Record Folder Contract
A submission folder must contain:
records/<track>/<submission-name>/
README.md
submission.json
train_gpt.py
train.log # after a real run
train_gpt.py must compile and run from inside this folder in a clean Parameter Golf checkout.
Competition Constraints To Preserve
Artifact cap: 16,000,000 decimal bytes.
Training cap for leaderboard records: 10 minutes on 8xH100 SXM-class hardware.
Evaluation metric: FineWeb validation bits per byte (val_bpb).
No validation-set leakage. Test-time training may only use validation tokens already scored, if implemented.
No hidden downloads/network calls during evaluation.
No local repository imports unless included and counted in the record folder.
If tokenizer changes, prove BPB accounting carefully; stock SP1024 is safest for first participation.
Self-Contained Script Checklist
Before running:
python -m py_compile train_gpt.py passes.
Imports are standard/allowed environment packages only (torch, numpy, sentencepiece, etc.).
DATA_PATH and TOKENIZER_PATH are env-configurable.
Script loads fineweb_train_*.bin and fineweb_val_*.bin with the Parameter Golf binary header format.
Script computes validation BPB from SentencePiece byte accounting, not just token loss.
Script logs parameter count and artifact-size estimate.
Script writes a compressed artifact, usually final_model.int8.ptz.
Script reloads/dequantizes the compressed artifact and evaluates the round-trip model.