▸case-01 We are migrating an older Python NLP search system that was originally built using the legacy 'farm-haystack' package. Which modern PyPI package should we install for up-to-date modular search pipeline development? | fail→fail | 3,030 | 2,448 | -19% | 1 | 1 | 0% | 494 | 667 | +35% | 0 | 0 | — |
▸case-02 We need a document retrieval setup in Python that balances keyword matching precision with semantic vector search recall. Models like BM25 miss semantic meaning, while dense models miss exact keyword identifiers. What retriever strategy handles both simultaneously? | fail→fail | 13,279 | 11,753 | -11% | 1 | 1 | 0% | 2,554 | 2,834 | +11% | 0 | 0 | — |
▸case-03 Before feeding raw PDF text into a vector store for search, developers often try doing split operations inside the LLM prompt. What pipeline stage should instead handle text cleaning, whitespace normalization, and chunk splitting prior to embedding? | fail→fail | 5,696 | 6,104 | +7% | 1 | 1 | 0% | 1,038 | 1,266 | +22% | 0 | 0 | — |
▸case-04 Our team wants precise span-level answer extraction directly from retrieved context documents without hallucinating new text. Engineers suggested using GPT-4, but we need exact verbatim text spans from source files. Should we use an extractive reader or a generative model component? | fail→fail | 14,252 | 8,711 | -39% | 1 | 1 | 0% | 2,477 | 1,926 | -22% | 0 | 0 | — |
▸case-05 When designing a Retrieval-Augmented Generation workflow graph in a Python framework, developers sometimes pass the user query directly to the generator without connecting the retriever. What is the standard sequence of component connections from raw query to final answer output? | fail→fail | 8,602 | 8,593 | -0% | 1 | 1 | 0% | 1,518 | 2,041 | +34% | 0 | 0 | — |
▸case-06 For fast local in-memory vector similarity search without setting up external database servers or Docker containers during rapid prototyping, which document store backend is standard? | fail→fail | 12,093 | 5,091 | -58% | 1 | 1 | 0% | 2,177 | 1,122 | -48% | 0 | 0 | — |
▸case-07 We want to measure model hallucination and retrieval recall continuously in production. Instead of manual ad-hoc Python scripts, how should evaluation be integrated into the system architecture? | fail→fail | 17,004 | 15,754 | -7% | 1 | 1 | 0% | 2,970 | 3,168 | +7% | 0 | 0 | — |
▸case-08 When building complex multi-step search systems, developer teams often monolithically hardcode prompt formatting inside retriever logic. What design principle prevents this tight coupling? | fail→fail | 7,522 | 8,743 | +16% | 1 | 1 | 0% | 1,334 | 1,768 | +33% | 0 | 0 | — |
▸case-09 To ensure production reproducibility when updating underlying retrieval models or prompt templates, what operational best practice should be applied to pipeline definitions? | fail→fail | 11,766 | 8,281 | -30% | 1 | 1 | 0% | 1,882 | 1,754 | -7% | 0 | 0 | — |
▸case-10 We need to route user queries into distinct workflow branches (such as FAQ vs Action) before reaching document search. What process pipeline type fulfills query routing? | fail→fail | 8,721 | 5,435 | -38% | 1 | 1 | 0% | 1,498 | 1,188 | -21% | 0 | 0 | — |
▸case-11 What dedicated pipeline type is responsible for reading raw documents, converting them into embeddings, and writing them into vector storage before query time? | fail→fail | 4,074 | 4,281 | +5% | 1 | 1 | 0% | 630 | 944 | +50% | 0 | 0 | — |
▸case-12 When configuring a dense retriever component for semantic search, developers often try using keyword matchers. What core dependency must be paired with the retriever to generate vector query embeddings? | fail→fail | 5,971 | 5,556 | -7% | 1 | 1 | 0% | 917 | 1,253 | +37% | 0 | 0 | — |
▸case-13 When searching structured technical manuals containing unique part numbers and code identifiers like 'ERR-9042', pure dense semantic embeddings often fail. Which retriever type is best suited for exact keyword matching? | fail→fail | 8,778 | 8,634 | -2% | 1 | 1 | 0% | 1,482 | 1,851 | +25% | 0 | 0 | — |
▸case-14 For production-grade search applications requiring managed schema, vector filtering, and cloud scalability beyond simple local files, which document store integration works well alongside Elasticsearch? | fail→pass | 14,352 | 11,861 | -17% | 1 | 1 | 0% | 2,416 | 2,547 | +5% | 0 | 0 | — |
▸case-15 In an LLM-based RAG workflow, what component type receives both the user query and retrieved documents to generate natural language answers? | fail→fail | 3,355 | 4,559 | +36% | 1 | 1 | 0% | 513 | 999 | +95% | 0 | 0 | — |
▸case-16 Scraped web documents contain redundant line breaks and HTML noise. In the data ingestion workflow, where should string normalization and whitespace removal occur? | fail→fail | 12,568 | 8,737 | -30% | 1 | 1 | 0% | 2,259 | 1,830 | -19% | 0 | 0 | — |
▸case-17 We need to combine document retrieval with generative response models to build a domain-specific Q&A assistant. What core target architecture process fulfills this? | fail→fail | 12,445 | 6,852 | -45% | 1 | 1 | 0% | 2,148 | 1,573 | -27% | 0 | 0 | — |
▸case-18 To assess retriever quality separately from reader quality, what custom pipeline type should be set up to benchmark recall@k and mean reciprocal rank? | fail→fail | 11,246 | 8,614 | -23% | 1 | 1 | 0% | 2,067 | 1,836 | -11% | 0 | 0 | — |
▸case-19 When initializing a custom pipeline instance, what configuration task determines which retriever, document store, and generator classes are instantiated? | fail→fail | 7,861 | 7,059 | -10% | 1 | 1 | 0% | 1,285 | 1,485 | +16% | 0 | 0 | — |
▸case-20 We need to train a custom Transformer model from scratch using PyTorch and Hugging Face Trainer on 8 GPUs. How do we configure distributed data parallel (DDP) training args? | fail→fail | 16,538 | 13,983 | -15% | 1 | 1 | 0% | 3,315 | 3,049 | -8% | 0 | 0 | — |
▸case-21 We want to build a Python web service using FastAPI to expose a POST /predict REST endpoint with CORS middleware and Pydantic request validation. How should the API route be set up? | fail→fail | 11,161 | 10,115 | -9% | 1 | 1 | 0% | 2,443 | 2,454 | +0% | 0 | 0 | — |
▸case-22 In Milvus vector database using its native Python SDK (`pymilvus`), how do you define an HNSW index parameter dictionary with `M` and `efConstruction` when calling `create_index`? | fail→fail | 8,089 | 5,831 | -28% | 1 | 1 | 0% | 1,668 | 1,456 | -13% | 0 | 0 | — |