Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/code-llm-papers-guide
Survey and paper collection on LLMs for code generation
Passed 18 of 22 test cases with the skill, and 11 of 22 without it.
and 9 more in the full report
7 fixed11 passed either way4 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.
For automated test suite evaluation on HumanEval, which models tracking pass@10 metrics show reported scores, and what is the recorded pass@10 score for GPT-4?
| Checked | Without skill | With skill |
|---|---|---|
| GPT-4 is recorded with a HumanEval pass@10 score of 86.0%. | Missed | Passed |
The instructions your model reads when the skill loads.
This curated collection covers LLMs for code — from foundational models (Codex, CodeGen, StarCoder) through code generation, completion, repair, translation, and understanding. Accompanies a TMLR survey paper providing systematic categorization. Tracks 500+ papers across pre-training, fine-tuning, evaluation, and application of code-focused language models.
Code LLMs
├── Pre-training
│ ├── Encoder-only (CodeBERT, GraphCodeBERT)
│ ├── Decoder-only (Codex, CodeGen, StarCoder, DeepSeek-Coder)
│ └── Encoder-Decoder (CodeT5, PLBART)
├── Fine-tuning & Alignment
│ ├── Instruction tuning (WizardCoder, Magicoder)
│ ├── RLHF for code (CodeRL)
│ └── Self-play (AlphaCode)
├── Applications
│ ├── Code generation (NL → Code)
│ ├── Code completion (infilling)
│ ├── Code repair (bug fixing)
│ ├── Code translation (language conversion)
│ ├── Code summarization (Code → NL)
│ ├── Test generation
│ └── Code review
└── Evaluation
├── Benchmarks (HumanEval, MBPP, SWE-bench)
├── Metrics (pass@k, CodeBLEU)
└── Security analysis
| Model | Year | Organization | Parameters | Key Innovation |
|---|---|---|---|---|
| CodeBERT | 2020 | Microsoft | 125M | Bimodal NL-PL pre-training |
| Codex | 2021 | OpenAI | 12B | GPT-3 fine-tuned on GitHub |
| AlphaCode | 2022 | DeepMind | 41B | Competitive programming |
| StarCoder | 2023 | BigCode | 15B | Fill-in-the-middle, 1T tokens |
test cases passed, vs 11 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.