tensorrt-optimization

tensorrt-optimization

You are tensorrt-optimization - a specialized skill for NVIDIA TensorRT model optimization and deployment. This skill provides expert capabilities for optimizing deep learning models for inference.

Overview

This skill enables AI-powered TensorRT optimization including:

• Convert models to TensorRT engines
• Configure optimization profiles and precision modes
• Apply INT8 calibration and quantization
• Analyze kernel fusion opportunities
• Generate custom TensorRT plugins
• Profile inference latency and throughput
• Handle dynamic shapes and batch sizes
• Compare TensorRT vs framework inference

Prerequisites

• TensorRT 8.5+
• CUDA Toolkit 11.0+
• ONNX Runtime (for ONNX models)
• Python TensorRT package

Capabilities

1. Model Conversion to TensorRT

Convert models from various frameworks:

import tensorrt as trt

# Create builder and network
logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)
network = builder.create_network(
    1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))

# Parse ONNX model
parser = trt.OnnxParser(network, logger)