▸case-01 We are building an intent classification service for customer support routing. The product team suggests waiting until we annotate 5,000 text samples per intent before training any model. What dataset size range per class is sufficient to achieve high accuracy using efficient contrastive text classification fine-tuning? | fail→pass | 12,566 | 7,186 | -43% | 1 | 1 | 0% | 2,190 | 1,431 | -35% | 0 | 0 | — |
▸case-02 Our team is designing a text classification system and spending weeks crafting complex system prompt templates and instructions for context windows. How does a contrastive few-shot framework handle text classification without prompt engineering? | fail→fail | 14,894 | 14,335 | -4% | 1 | 1 | 0% | 2,461 | 2,603 | +6% | 0 | 0 | — |
▸case-03 We need to train 20 distinct task-specific intent classifiers daily on a standard GPU server. Should we expect training jobs for each lightweight few-shot sentence transformer model to take several hours or days? | fail→fail | 13,008 | 7,682 | -41% | 1 | 1 | 0% | 2,286 | 1,640 | -28% | 0 | 0 | — |
▸case-04 When choosing a foundation model for efficient few-shot sentence classification under tight latency constraints, should we select a 7B autoregressive decoder LLM or a sentence transformer encoder model? | fail→fail | 15,079 | 10,174 | -33% | 1 | 1 | 0% | 2,526 | 2,002 | -21% | 0 | 0 | — |
▸case-05 In a two-stage few-shot classification pipeline, what fine-tuning technique is applied to the base embedding model in the first stage before fitting the classification head? | fail→pass | 8,704 | 3,478 | -60% | 1 | 1 | 0% | 1,621 | 819 | -49% | 0 | 0 | — |
▸case-06 After contrastive fine-tuning generates dense text representations, what component is trained on top of the fixed or fine-tuned embeddings to output class logits? | fail→fail | 5,135 | 4,138 | -19% | 1 | 1 | 0% | 883 | 839 | -5% | 0 | 0 | — |
▸case-07 In the first stage of SetFit training, contrastive fine-tuning is performed on generated sentence pairs. Which specific loss class from sentence-transformers is used by default to calculate similarity between positive and negative pair embeddings? | fail→fail | 5,059 | 2,891 | -43% | 1 | 1 | 0% | 788 | 730 | -7% | 0 | 0 | — |
▸case-08 When configuring SetFit training, a developer attempts to increase pair sampling density by passing `sample_size=50`, but encounters an invalid parameter error. Which hyperparameter controls the number of positive and negative sentence pairs generated per sample? | fail→fail | 5,334 | 6,169 | +16% | 1 | 1 | 0% | 1,025 | 1,334 | +30% | 0 | 0 | — |
▸case-09 When training a SetFit model where GPU acceleration and PyTorch gradient descent across the classification head are required, which argument replaces the default scikit-learn LogisticRegression classifier with a neural network head? | fail→fail | 5,367 | 4,451 | -17% | 1 | 1 | 0% | 976 | 1,061 | +9% | 0 | 0 | — |
▸case-10 We want to implement efficient contrastive few-shot intent classification in Python. Which primary open-source framework specifically combines sentence-transformers with scikit-learn style classification heads for this purpose? | fail→fail | 6,073 | 7,449 | +23% | 1 | 1 | 0% | 1,132 | 1,595 | +41% | 0 | 0 | — |
▸case-11 What key underlying library provides the base embedding models and contrastive pair loss implementations used by few-shot sentence classification frameworks? | fail→fail | 9,208 | 3,114 | -66% | 1 | 1 | 0% | 1,401 | 716 | -49% | 0 | 0 | — |
▸case-12 Our engineering group is replacing a legacy rule-based regex router with an automated ML pipeline. What specific dialogue management subsystem or routing pipeline is best suited for few-shot sentence classification? | fail→fail | 17,707 | 10,324 | -42% | 1 | 1 | 0% | 2,619 | 2,104 | -20% | 0 | 0 | — |
▸case-13 When serving a trained few-shot sentence classifier on edge instances or API workers, what parameter scale and deployment footprint can we expect compared to hosting full 13B parameter LLMs? | fail→fail | 19,689 | 12,916 | -34% | 1 | 1 | 0% | 3,569 | 2,500 | -30% | 0 | 0 | — |
▸case-14 What specific hyperparameter controls how many passes over positive and negative generated text pairs are performed during the embedding adaptation stage? | fail→fail | 12,526 | 7,049 | -44% | 1 | 1 | 0% | 2,182 | 1,421 | -35% | 0 | 0 | — |
▸case-15 How does contrastive fine-tuning construct sufficient text comparison data from a small set of 8 labeled examples per class? | fail→pass | 13,999 | 11,453 | -18% | 1 | 1 | 0% | 2,370 | 2,410 | +2% | 0 | 0 | — |
▸case-16 What types of classifier heads are standardly supported on top of sentence embeddings for few-shot intent determination? | fail→fail | 15,954 | 10,836 | -32% | 1 | 1 | 0% | 2,702 | 2,180 | -19% | 0 | 0 | — |
▸case-17 Why might a production intent classifier favor a fine-tuned sentence transformer model trained on 10 examples over a zero-shot LLM API call? | fail→fail | 12,904 | 12,464 | -3% | 1 | 1 | 0% | 2,249 | 2,334 | +4% | 0 | 0 | — |
▸case-18 Compared to standard full fine-tuning of BERT which requires hundreds of samples per class to avoid overfitting, how many samples per class does contrastive embedding adaptation require? | fail→pass | 6,074 | 3,092 | -49% | 1 | 1 | 0% | 959 | 711 | -26% | 0 | 0 | — |
▸case-19 How does training speed affect prompt-less few-shot classifiers when experimenting with new intent categories in production? | fail→fail | 15,067 | 13,096 | -13% | 1 | 1 | 0% | 2,423 | 2,343 | -3% | 0 | 0 | — |
▸case-20 We have 100,000 multi-turn conversation logs and want to perform full parameter fine-tuning of Llama-3 70B for general instruction following across GPUs. What framework and methodology should we use? | fail→fail | 22,575 | 16,407 | -27% | 1 | 1 | 0% | 3,591 | 3,380 | -6% | 0 | 0 | — |
▸case-21 We want to construct a dynamic prompt template that injects 3 few-shot query-response examples into an OpenAI GPT-4o chat completion payload at runtime. How should we format the system and user messages? | fail→fail | 11,983 | 12,776 | +7% | 1 | 1 | 0% | 2,404 | 2,709 | +13% | 0 | 0 | — |
▸case-22 We need to index 5 million PDF documentation chunks into an HNSW vector index using Qdrant for semantic similarity search in a RAG pipeline. How should we set up distance metrics and payload storage? | fail→fail | 20,205 | 15,232 | -25% | 1 | 1 | 0% | 3,256 | 3,115 | -4% | 0 | 0 | — |