▸case-06 We are deploying a PyTorch transformer model to a latency-critical C++ microservice where we want maximum single-node inference throughput without PyTorch runtime overhead. A colleague suggested saving the raw model using pickle and loading it in python via Flask. What export format and runtime engine should we adopt instead, and how do we convert the model? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-07 We are training an 8-billion parameter language model on 4 GPUs with 24GB VRAM each. Standard DistributedDataParallel fails with CUDA Out of Memory errors during backward passes because full model parameters and optimizer states exceed 24GB per GPU. What PyTorch distributed strategy should we implement to shard model states across GPUs? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-18 We need to upgrade our production Kubernetes deployment of TorchServe from version 1.2 to 1.3 without dropping any incoming HTTP inference requests during the transition. Which deployment strategy should we configure in Kubernetes? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-15 Our organization struggles with model reproducibility because data science teams deploy raw `.pkl` files to S3 without tracking which Git commit, dataset version, or hyperparameter set produced the artifact. What system should we implement to manage model lifecycle, versioning, and lineage? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-13 Our e-commerce ranking model experiences prediction accuracy decay during flash sales due to shifting user feature distributions. We want real-time alerts when input feature distributions diverge from training distributions. What data validation tool or statistical method should we integrate into our streaming data pipeline? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-14 Training our ResNet-101 model on NVIDIA V100 GPUs using standard FP32 takes 36 hours. We want to double training speed and reduce memory consumption while maintaining model convergence without altering model layer counts. What PyTorch utility should we enable? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-09 We are running a 500-trial hyperparameter sweep for a deep neural network on PyTorch. Currently, each trial runs for a fixed 100 epochs, consuming excessive GPU hours even when early validation loss indicates poor hyperparameter choices. What optimization framework and pruning scheduler should we use? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-12 We need to score 500 million customer record vectors daily for churn prediction using an XGBoost model. Running a single Python script with Pandas and scikit-learn runs out of RAM and takes 18 hours. What distributed processing framework should we use to parallelize batch prediction across a cluster? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-01 Our team needs to set up model drift monitoring and an automated retraining pipeline for a tabular classification service. Can you provide an architectural layout and code examples showing how to detect data drift, trigger retraining workflows upon performance degradation, and log operational metrics for monitoring dashboards? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-02 I need an enterprise ML feature pipeline architecture that can serve both real-time online features for fast API requests and batch offline features for model training. Please generate a detailed system design document along with setup instructions, testing strategies for data quality, and operational guidelines for production deployment. | pass→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-05 Write a PostgreSQL query to calculate the 30-day active user retention rate grouped by signup cohort month and acquisition channel for our product analytics team. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-21 Our credit risk model degrades in accuracy every few months. Rather than manually re-running Jupyter notebooks, we need a production workflow orchestrator that automatically extracts fresh data, retrains the model, evaluates validation metrics against a threshold, and promotes the model if it passes. What orchestrator and workflow design should we build? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-20 We are building a semantic search engine for 10 million text documents. Storing raw 768-dimensional float embeddings in MongoDB and computing cosine similarity with a Python loop takes 8 seconds per query. What specialized database engine should we migrate to for sub-50ms vector search? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-17 Our team wants to block pull requests to our ML repository if code changes break model prediction schema or cause latency to exceed 50ms on sample benchmark inputs. What CI pipeline testing strategy should we implement? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-10 Our online fraud detection API requires reading 50 real-time feature values per request with sub-10ms P99 latency. Storing these features in PostgreSQL causes slow read times under peak load of 5,000 QPS. What dedicated feature store architecture and storage engine should we deploy? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-08 Our computer vision edge model running on CPU has a latency of 120ms per image. We want to reduce inference latency under 40ms without retraining the model parameters from scratch. What model optimization technique should we apply using PyTorch or ONNX Runtime? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-03 We have a CSV export of customer survey responses with 15 columns of Likert-scale ratings and text feedback. Please perform a statistical summary analysis calculating means, medians, and correlation matrices to identify customer satisfaction trends for our quarterly management report. | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-04 I need a React component using Tailwind CSS that renders a responsive grid of monitoring cards displaying CPU percentage, memory usage, and request latency with a togglable dark mode. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-11 We are updating our live recommendation model from v1 to v2. To prevent revenue drop from potential performance regressions, we want to gradually route 10% of production traffic to v2 while monitoring click-through rates against v1 before full rollout. What deployment strategy and load balancer configuration should we use? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-16 We are replacing a slow Python Flask inference wrapper that blocks on synchronous requests. We need a high-throughput microservice in Python capable of handling 2,000 asynchronous prediction requests per second with strict payload validation. What modern API framework and concurrency pattern should we adopt? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-19 We want to domain-adapt a 7B parameter base LLM on a single consumer GPU with 16GB VRAM. Full fine-tuning causes immediate CUDA OOM. What parameter-efficient fine-tuning technique and memory optimization library should we use? | pass→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-22 Our automated loan approval system was flagged by compliance regulators who require clear explanations of which financial features contributed most to each applicant's rejection score. What model interpretability framework should we integrate into our serving pipeline? | pass→pass | — | — | — | — | — | — | — | — | — | — | — | — |