Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mkurman/model-compression
Model compression techniques: pruning, knowledge distillation, and quantization. Covers compression ordering (P-KD-Q), tools, and evaluation metrics.
Passed 19 of 22 test cases with the skill, and 16 of 22 without it.
and 4 more in the full report
5 fixed14 passed either way1 still failing2 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 5 test cases the skill fixed. Same model, same prompt, with and without the skill.
In empirical benchmarks on Qwen2.5 3B, what overall compression ratio is achieved using the P-KD-Q methodology while retaining language understanding?
| Checked | Without skill | With skill |
|---|---|---|
| states that a 3.68x compression ratio is achieved | Missed | Passed |
The instructions your model reads when the skill loads.
Model compression reduces LLM size and inference cost through three main techniques: pruning (removing parameters), knowledge distillation (training smaller student models), and quantization (lower precision weights). A 2025 study on Qwen2.5 3B found the optimal ordering: Pruning → Knowledge Distillation → Quantization (P-KD-Q) achieves 3.68x compression while preserving instruction-following and language understanding.
| Technique | What it does | Compression | Quality Impact |
|---|---|---|---|
| Quantization | Lower precision (FP16→INT4/FP8) | Highest standalone | Low with PTQ, recoverable with QAT |
| Structured Pruning | Remove attention heads, layers, neurons | Moderate | Moderate degradation, recoverable with KD |
| Knowledge Distillation | Train smaller model on larger model outputs | Architecture-dependent | Best quality recovery post-pruning |
The paper shows ordering matters critically:
Applying quantization early causes irreversible information loss that impairs subsequent training.
| Toolkit | Features | Use Case |
|---|---|---|
angelslim (Tencent) | PTQ/QAT, pruning, Eagle3 speculative decoding | Full compression pipeline, 1.25-bit to FP8 |
nvidia-model-optimizer | PTQ, QAT, pruning, distillation, speculative decoding | NVIDIA ecosystem, TensorRT-LLM deployment |
intel-neural-compressor | INT8/FP8/INT4 quantization, pruning, distillation | Intel hardware, ONNX Runtime |
peft + unsloth | QLoRA fine-tuning | Training-time efficiency, adapter-based |
test cases passed, vs 16 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.