LLMOps – Inference & Optimization – Production Skill Hub

LLMOps – Inference & Optimization – Production Skill Hub

Modern Best Practices (January 2026):

• Treat inference as a systems problem: SLOs, tail latency, retries, overload, and cache strategy.
• Use continuous batching / smart scheduling when serving many concurrent requests (Orca scheduling: https://www.usenix.org/conference/osdi22/presentation/yu).
• Use KV-cache aware serving (PagedAttention/vLLM: https://arxiv.org/abs/2309.06180) and efficient attention kernels (FlashAttention: https://arxiv.org/abs/2205.14135).
• Use speculative decoding when latency is critical and draft-model quality is acceptable (speculative decoding: https://arxiv.org/abs/2302.01318).
• Quantize only with measured quality impact and rollback plan (quantization must be validated on your eval set).

This skill provides production-ready operational patterns for optimizing LLM inference performance, cost, and reliability. It centralizes decision rules, optimization strategies, configuration templates, and operational checklists for inference workloads.

No theory. No narrative. Only what Claude can execute.

When to Use This Skill

Claude should activate this skill whenever the user asks for:

• Optimizing LLM inference latency or throughput
• Choosing quantization strategies (FP8/FP4/INT8/INT4)
• Configuring vLLM, TensorRT-LLM, or DeepSpeed inference
• Scaling LLM inference across GPUs (tensor/pipeline parallelism)
• Building high-throughput LLM APIs
• Improving context window performance (KV cache optimization)
• Using speculative decoding for faster generation
• Reducing cost per token
• Profiling and benchmarking inference workloads
• Planning infrastructure capacity
• CPU/edge deployment patterns
• High availability and resilience patterns

Scope Boundaries (Use These Skills for Depth)

• Prompting, tuning, datasets → ai-llm
• RAG pipeline construction → ai-rag
• Deployment, APIs, monitoring → ai-mlops
• Safety, governance → ai-mlops