Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/cikm-experiments
Use when designing or auditing the empirical program of a CIKM paper — matching evidence to the claim's lanes across retrieval, mining, and knowledge-management evaluation cultures, choosing datasets and baselines that survive a blended panel, isolating the boundary mechanism, and meeting applied-track deployment-evidence bars.
Passed 14 of 21 test cases with the skill, and 11 of 21 without it.
and 6 more in the full report
5 fixed10 passed either way6 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
I'm prepping a submission on neural retrieval augmented with knowledge graphs for CIKM and want to audit my experimental section. Could you review my plan and produce a report with: 1) a contract mapping each claim in my paper to its evaluation lane, required evidence, and current status; 2) a clear definition of the decisive ablation specifying what component is removed while keeping everything else fixed; 3) an assessment of which reviewer lane or community still lacks dataset coverage; 4) a list of missing reporting floor items for my tables, captions, and artifacts; and 5) a recommendation for the single highest-priority experiment to run next?
| Checked | Without skill | With skill |
|---|---|---|
| Uses output section tags [Contract table], [Decisive ablation], [Panel coverage], [Reporting floor], and [Next run] | Missed | Passed |
| Recommends the decisive ablation as the single experiment to schedule first | Missed | Passed |
The instructions your model reads when the skill loads.
The empirical program of a CIKM paper answers to three evaluation cultures. IR culture wants ranked-metric discipline on recognized collections with significance testing; mining culture wants mechanism isolation, scaling behavior, and honest baselines; KM/database culture wants evidence that the method survives real data — heterogeneous, noisy, incomplete. Design the experiment section as the union of what the claimed lanes require, not the maximum of one.
For every claim sentence in the introduction, write down which lane it invokes and what that lane's standard evidence is:
| Claim invokes... | Required evidence pattern | Frequent hole |
|---|---|---|
| Better retrieval/ranking | Standard collections, tuned baselines, metric@k with paired significance tests | Untuned baselines; "significant" without a test |
| A better mechanism | Ablation removing exactly that mechanism; sensitivity to its key parameter | Ablations that vary three things at once |
| Scalability | Time/memory vs. input-size curves on stated hardware | A single wall-clock number, no curve |
| Robust to real data | Noise/incompleteness injection or a genuinely messy dataset | Only clean benchmark data |
| KG/side-information helps | The identical model minus the KG, plus a KG-quality sweep | Confounding KG addition with capacity increase |
| Deployed impact (Applied track) | Launch evidence: online metrics, A/B or pre/post windows, traffic scale | Offline proxy metrics presented as deployment proof |
The Applied Research row is a track requirement, not a style preference: CIKM 2026's call demands substantiation by a system launch, data release, or equivalent practical evidence.
test cases passed, vs 11 / 21 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.