▸case-01 I am configuring a Triton Protobuf configuration file (`config.pbtxt`) for a PyTorch ResNet50 model. I want to enable dynamic batching to increase throughput, but I want to ensure requests aren't held forever if traffic is light. A colleague suggested just setting `preferred_batch_size: [ 8, 16 ]` without any delay limit. How should I configure dynamic batching to enforce a maximum delay of 500 microseconds before executing a batch? | pass→pass | 9,038 | 11,251 | +24% | 1 | 1 | 0% | 1,736 | 2,185 | +26% | 0 | 0 | — |
▸case-02 I am setting up a `config.pbtxt` for an ONNX model that processes input images of shape `[3, 224, 224]`. The model accepts batching on the first dimension. Someone told me to include the batch dimension inside the `dims` array of the input configuration (e.g., `dims: [ -1, 3, 224, 224 ]`) while setting `max_batch_size: 16`. How should `max_batch_size` and the input `dims` field be defined according to Triton Protobuf specification? | pass→pass | 15,014 | 11,697 | -22% | 1 | 1 | 0% | 2,422 | 2,733 | +13% | 0 | 0 | — |
▸case-03 I have 4 GPUs on my inference host and want Triton Inference Server to deploy 2 instances of my transformer model on GPU 0 and GPU 1. A junior engineer suggested creating 4 separate model directories. How should `instance_group` be configured in a single `config.pbtxt` to explicitly assign instances across specific GPU IDs? | pass→pass | 11,396 | 10,225 | -10% | 1 | 1 | 0% | 2,209 | 2,256 | +2% | 0 | 0 | — |
▸case-04 I am deploying an LLM audio transcriber backend in Triton Inference Server that streams intermediate output tokens asynchronously back to the client over gRPC before final completion. I was advised to use standard unary responses. How should the transaction policy be configured in `config.pbtxt` to support multiple responses per single request? | pass→pass | 16,621 | 14,175 | -15% | 1 | 1 | 0% | 2,489 | 2,956 | +19% | 0 | 0 | — |
▸case-05 I need to chain a pre-processing Python model and an ONNX vision model together inside Triton Inference Server into a single inference endpoint named `vision_pipeline`. Someone suggested writing a custom C++ backend wrapper to call them sequentially. What is the standard Triton configuration structure for orchestrating this without custom host backend code? | pass→fail | 13,655 | 16,728 | +23% | 1 | 1 | 0% | 2,715 | 3,019 | +11% | 0 | 0 | — |
▸case-06 I am serving an RNN-based speech recognition model in Triton Inference Server where subsequent requests from the same user session share state. A team member suggested using standard `dynamic_batching`. What configuration block must be used instead in `config.pbtxt` to route requests with matching correlation IDs to the same model sequence instance? | pass→pass | 7,640 | 11,343 | +48% | 1 | 1 | 0% | 1,415 | 2,371 | +68% | 0 | 0 | — |
▸case-07 I want Triton Inference Server to cache responses for identical input payloads to avoid redundant inference calls on a BERT embeddings model. Someone said Triton requires Redis for caching response payloads. How do you enable Triton's built-in response caching directly in the `config.pbtxt`? | pass→pass | 10,176 | 12,547 | +23% | 1 | 1 | 0% | 1,770 | 2,714 | +53% | 0 | 0 | — |
▸case-08 I have an ONNX model served via the `onnxruntime_onnx` backend in Triton Inference Server, and I want Triton to automatically optimize execution using TensorRT execution provider on NVIDIA GPUs. A colleague recommended converting the ONNX file manually to `.plan` using `trtexec` beforehand. How can this be enabled directly in `config.pbtxt` using accelerator configuration? | pass→pass | 12,237 | 17,153 | +40% | 1 | 1 | 0% | 2,335 | 2,986 | +28% | 0 | 0 | — |
▸case-09 I am serving a PyTorch model using Triton's Python backend (`backend: "python"`). The script depends on specific Python libraries installed in a conda tarball named `myenv.tar.gz` stored in the model directory. Someone suggested setting `PYTHONPATH` system-wide. How should the conda environment archive path be specified inside `config.pbtxt` under `parameters`? | pass→pass | 11,741 | 13,120 | +12% | 1 | 1 | 0% | 2,175 | 2,373 | +9% | 0 | 0 | — |
▸case-10 To avoid cold-start latency spikes when new instances of my object detector load into Triton Inference Server, I want to run model warmup with sample input tensors upon loading. A developer suggested sending dummy gRPC requests from an external script right after boot. How can model warmup with zero-filled synthetic input tensors be configured directly inside `config.pbtxt`? | pass→pass | 11,937 | 14,293 | +20% | 1 | 1 | 0% | 2,316 | 3,160 | +36% | 0 | 0 | — |
▸case-11 My model repository folder contains subdirectories `1`, `2`, and `3`. By default, Triton Inference Server serves version `3`. I want Triton to serve both version `2` and `3` concurrently, but exclude version `1`. Someone suggested deleting folder `1`. How can this be controlled declaratively in `config.pbtxt` using version policy? | pass→pass | 8,336 | 10,167 | +22% | 1 | 1 | 0% | 1,541 | 2,325 | +51% | 0 | 0 | — |
▸case-12 I am organizing my local directory layout for Triton Inference Server for a model named `text_classifier`. I placed the config file at `text_classifier/config.pbtxt` and the ONNX model file at `text_classifier/model.onnx`. Triton throws a model loading error. What exact directory structure is required for model versioning in Triton's model repository? | pass→pass | 6,409 | 6,890 | +8% | 1 | 1 | 0% | 1,333 | 1,754 | +32% | 0 | 0 | — |
▸case-13 I need a Triton Python backend model to programmatically call another model deployed on the same Triton server, process its tensor outputs, and return a final result. A colleague suggested sending HTTP requests to `localhost:8000` from inside the Python backend script. What native Triton Python API mechanism should be used for in-process model execution? | pass→pass | 10,065 | 15,170 | +51% | 1 | 1 | 0% | 1,884 | 3,410 | +81% | 0 | 0 | — |
▸case-14 I am configuring input and output tensors in `config.pbtxt` for a text-generation backend that outputs variable-length UTF-8 strings. A user tried using `data_type: TYPE_STRING` but declared `dims: [ -1, 256 ]` with fixed byte allocation. How should variable-length string tensors be configured in terms of `data_type` and `dims`? | pass→pass | 13,006 | 14,432 | +11% | 1 | 1 | 0% | 2,440 | 3,100 | +27% | 0 | 0 | — |
▸case-15 I want to limit the rate at which requests are scheduled on a specific model instance in Triton Inference Server to avoid exceeding a shared GPU memory limit. A teammate suggested inserting `time.sleep()` inside the model execution script. How is rate limiting for model instances configured in Triton's `config.pbtxt`? | pass→pass | 12,942 | 13,716 | +6% | 1 | 1 | 0% | 2,333 | 2,812 | +21% | 0 | 0 | — |
▸case-16 My ONNX model file expects a 4D input tensor `[ batch, channels, height, width ]`, but the client sends a flat 1D byte vector that Triton needs to reshape before passing to the backend model. Someone suggested writing a custom Python client to reshape the bytes before sending. How can `dims` and `reshape` fields in `config.pbtxt` handle this mapping? | pass→pass | 17,375 | 17,993 | +4% | 1 | 1 | 0% | 3,183 | 3,625 | +14% | 0 | 0 | — |
▸case-17 I have two models running on Triton Inference Server: `fraud_detect` (latency-critical) and `batch_report` (background). I want Triton's dynamic batch scheduler to prioritize queue processing for `fraud_detect` over `batch_report`. Someone told me Triton doesn't support model-level priority settings. How do you set scheduling priority in `config.pbtxt`? | fail→pass | 15,819 | 23,413 | +48% | 1 | 1 | 0% | 2,859 | 4,660 | +63% | 0 | 0 | — |
▸case-18 I am creating a minimal `config.pbtxt` for a TensorRT model `resnet50.plan`. I don't want to manually specify all 100 layer tensor names in `input` and `output` blocks. A colleague claimed Triton requires complete explicit specification for every model. What parameter or feature allows Triton to automatically derive input and output tensor shapes directly from the TensorRT engine? | pass→pass | 9,644 | 9,340 | -3% | 1 | 1 | 0% | 1,694 | 2,006 | +18% | 0 | 0 | — |
▸case-19 I want to disable summary metrics collection for a high-frequency internal ping model in Triton Inference Server to reduce Prometheus metric overhead. A developer suggested filtering the Prometheus scraper output. How can model metric collection be toggled directly in Triton `config.pbtxt`? | fail→pass | 12,413 | 16,003 | +29% | 1 | 1 | 0% | 2,040 | 3,150 | +54% | 0 | 0 | — |
▸case-20 I am building a PyTorch ResNet50 neural network model for image classification. How do I define the `forward` method in Python to handle cross-entropy loss calculation during training? | pass→pass | 12,785 | 15,713 | +23% | 1 | 1 | 0% | 2,475 | 2,819 | +14% | 0 | 0 | — |
▸case-21 I am deploying Triton Inference Server on a Kubernetes cluster using Helm. I need an NGINX Ingress controller resource manifest in YAML format to route gRPC traffic on port 8001 and HTTP traffic on port 8000 to the `triton-service`. How do I structure the Kubernetes Ingress YAML? | pass→pass | 12,980 | 12,877 | -1% | 1 | 1 | 0% | 2,735 | 3,002 | +10% | 0 | 0 | — |
▸case-22 I have a trained PyTorch model object `model` in Python. I want to export this model to ONNX format on disk as `model.onnx` with dynamic batch size on the first dimension using `torch.onnx.export`. What is the Python script to perform this export? | pass→pass | 10,250 | 6,186 | -40% | 1 | 1 | 0% | 2,057 | 1,601 | -22% | 0 | 0 | — |
▸case-23 I am launching an instance of `vllm.entrypoints.openai.api_server` from CLI to serve Llama 3 on 2 GPUs with tensor parallelism. What CLI flags should I pass to enable tensor parallelism of 2 and GPU memory utilization of 0.9? | pass→pass | 7,044 | 6,247 | -11% | 1 | 1 | 0% | 1,483 | 1,530 | +3% | 0 | 0 | — |