Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/colt-experiments
Use when deciding whether a COLT (Conference on Learning Theory) paper needs numerical content at all — COLT has no experiments requirement — and, when numerics genuinely help, designing small illustrative simulations that visualize a proved bound, a separation, or a phase transition without diluting the theory.
Passed 20 of 23 test cases with the skill, and 11 of 23 without it.
and 10 more in the full report
10 fixed10 passed either way2 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
List all necessary elements that must be included in a COLT figure caption for a synthetic rate plot. Structure your answer using the standard 5 bracketed block format.
| Checked | Without skill | With skill |
|---|---|---|
| The response requires stating the replication count in the figure caption. | Passed | Passed |
| The response requires including a single-clause takeaway in the figure caption. | Missed | Passed |
The instructions your model reads when the skill loads.
Start from the venue truth: COLT imposes no experiments requirement, and most COLT papers contain no experiments. The 2026 CFP (checked 2026-07-08) asks for theoretical machine-learning contributions and mentions no empirical-evaluation expectation; papers are accepted on theorems. The decision this skill supports is therefore whether to include numerics, and only then how.
| Situation | Include numerics? | Rationale |
|---|---|---|
| Clean upper/lower bound pair, standard model | No | Plots add length, not belief |
| New algorithm whose practicality is part of the pitch | Small illustration | Shows the constants are not absurd |
| Theory explaining an empirical phenomenon (in-scope per the CFP's inclusive view) | Yes, essential | The phenomenon must be exhibited, then explained |
| Conjectured tightness you cannot prove | Careful, labeled | A scaling plot can support a conjecture — never upgrade it |
| Phase transition / separation between models | Often worthwhile | A picture of the transition is the most readable evidence |
| Purely structural result (equivalences, characterizations) | No | Nothing to simulate |
If the answer is no, spend the pages on proof overviews instead; a decorative benchmark table in a COLT submission signals venue confusion and can lower reviewer confidence.
The canonical COLT figure is a log-log rate check — empirical error or regret against
test cases passed, vs 11 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.