Post-training quantization

Overview

Intel Neural Compressor provides low-bit quantization (INT8, FP8, INT4, MXFP4, NVFP4), sparsity, pruning, and knowledge distillation for optimizing models on Intel hardware and beyond.

Installation

uv pip install neural-compressor

Basic Quantization

from neural_compressor import Quantization, config

# Post-training quantization
quantizer = Quantization(config)
q_model = quantizer(model)
q_model.save("quantized_model")

Pruning

from neural_compressor import Pruning

pruner = Pruning(model, config={"pruning_type": "snip_momentum", "target_sparsity": 0.3})
pruned_model = pruner.fit()

References

• Intel NC docs
• Intel NC GitHub