---
name: mkurman-model-pruning
description: "Structured and unstructured model pruning: weight pruning, attention head pruning, layer removal, and neural architecture search for efficient LLMs."
license: "MIT"
metadata:
  decimalai-skill-id: "3f6f4306-c4a6-4b84-b358-b7c589da10d7"
  decimalai-version-id: "6c900649-a4f0-40a8-b751-5ace9c313585"
  decimalai-version: "1"
  decimalai-body-sha256: "e76933d0b608b039d1d2a59e4de41d4d8c71ad4e42117750c08570b1a89aa20a"
  decimalai-name: "mkurman/model-pruning"
  decimalai-format: "1"
  decimalai-invocation: "model"
  decimalai-category: "ai-ml"
  decimalai-source: "https://app.decimal.ai/s/mkurman-model-pruning@1/SKILL.md"
---
## Overview

Pruning removes redundant parameters from neural networks. **Structured pruning** removes entire heads/layers (hardware-friendly), while **unstructured pruning** sets individual weights to zero (higher compression, needs sparse hardware support).

## Pruning Types

| Type | What's removed | Hardware Support | Compression |
|---|---|---|---|
| Unstructured (N:M sparsity) | Individual weights | NVIDIA 2:4 sparse cores | 2x (2:4) |
| Structured — Head pruning | Attention heads | Any GPU | 1.5-3x |
| Structured — Layer pruning | Transformer layers | Any GPU | 1.5-5x |
| Structured — Neuron pruning | FFN neurons | Any GPU | 1.5-2x |

## Example: Unstructured Pruning with SparseGPT

```python
from transformers import AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")

# Apply SparseGPT (one-shot, no training needed)
from sparsegpt import SparseGPT
sparsifier = SparseGPT(model)
sparsifier.compress(sparsity=0.5)  # 50% weight sparsity

# Or use NVIDIA 2:4 sparsity
for name, param in model.named_parameters():
    if param.dim() >= 2:
        param.data = torch.nn.utils.parametrizations.prune_2in4(param.data)
```

## Structured Pruning

```python
# Layer removal example
def prune_layers(model, layers_to_keep):
    model.model.layers = model.model.layers[:layers_to_keep]
    model.config.num_hidden_layers = layers_to_keep
    return model

# After pruning, re-train or distill to recover quality
```

## References
- [SparseGPT paper](https://arxiv.org/abs/2301.00774)
- [LLM Pruning survey](https://arxiv.org/abs/2406.13794)
- [NVIDIA 2:4 sparsity](https://developer.nvidia.com/blog/achieving-fp32-inference-performance-with-sparse-float-core-on-ampere/)