▸case-01 We are building a code search assistant that needs to index Python and TypeScript repositories into a vector database. Please design a chunking strategy and provide code that splits source files intelligently along language structures rather than arbitrary line counts. | fail→pass | 24,875 | 20,196 | -19% | 1 | 1 | 0% | 5,301 | 4,282 | -19% | 0 | 0 | — |
▸case-02 We have a large knowledge base of technical manuals written in Markdown with deep heading hierarchies (#, ##, ###). Developers often get fragmented answers because headings are divorced from content when using standard character slicing. What chunking component should we use to split these files while preserving header context? | fail→fail | 12,467 | 7,298 | -41% | 1 | 1 | 0% | 2,103 | 1,658 | -21% | 0 | 0 | — |
▸case-03 Our RAG system processes unstructured narrative reports where paragraphs vary wildly in length and topic shifts happen unpredictably. Fixed-size splitting splits sentences in half, while paragraph splitting creates 2000-word blocks. We want chunks to split dynamically where the semantic meaning changes. How should we implement this in Python? | fail→fail | 21,140 | 19,775 | -6% | 1 | 1 | 0% | 3,797 | 3,393 | -11% | 0 | 0 | — |
▸case-04 We are ingesting raw plain text documents into a vector search database. We want a text splitter that tries to keep paragraphs together, falling back to sentences, then words, and finally characters if necessary to fit within a target size. What standard class handles this separator hierarchy? | fail→fail | 5,312 | 6,596 | +24% | 1 | 1 | 0% | 1,068 | 1,207 | +13% | 0 | 0 | — |
▸case-05 We are using an embedding API that strictly limits inputs to 512 tokens. Character-based splitting is causing frequent HTTP 400 context length errors because token length varies per character. What text splitting strategy directly measures token counts instead of character lengths? | pass→pass | 11,686 | 8,975 | -23% | 1 | 1 | 0% | 2,070 | 2,027 | -2% | 0 | 0 | — |
▸case-06 Our team is setting up a new Python RAG ingestion microservice. We want to standardize on an established, maintained library that provides built-in text splitters for recursive, token, markdown, and code chunking rather than writing custom splitting functions. What standard Python package should we add to our requirements.txt? | fail→fail | 6,906 | 4,464 | -35% | 1 | 1 | 0% | 1,414 | 1,179 | -17% | 0 | 0 | — |
▸case-07 When setting up semantic distance-based chunking for text documents in Python, what underlying library handles the local sentence embedding calculations required to compute semantic boundaries? | fail→fail | 8,777 | 4,392 | -50% | 1 | 1 | 0% | 1,553 | 1,080 | -30% | 0 | 0 | — |
▸case-08 When configuring chunk size for a document indexing pipeline, what is the key constraint regarding the downstream embedding model that determines the upper bound for the chunk size? | fail→fail | 5,475 | 4,991 | -9% | 1 | 1 | 0% | 959 | 1,038 | +8% | 0 | 0 | — |
▸case-09 During chunking configuration, search queries frequently fail to match facts that happen to cross the boundary between chunk N and chunk N+1. What configuration parameter must be set to preserve context between contiguous chunks? | fail→fail | 3,296 | 3,238 | -2% | 1 | 1 | 0% | 555 | 808 | +46% | 0 | 0 | — |
▸case-10 We have a heterogeneous document pipeline containing Markdown guides, Python scripts, and unstructured prose. Instead of applying a single generic character splitter to all files, what general principle should govern how we select a chunking strategy for each file? | fail→fail | 9,433 | 8,122 | -14% | 1 | 1 | 0% | 1,587 | 1,724 | +9% | 0 | 0 | — |
▸case-11 Our team has implemented recursive splitting with 500 characters and 50 overlap, but stakeholders complain that retrieval accuracy is inconsistent. Before changing our embedding model or vector database, how should we systematically determine the optimal chunk size and strategy? | fail→fail | 15,147 | 15,895 | +5% | 1 | 1 | 0% | 2,627 | 3,191 | +21% | 0 | 0 | — |
▸case-12 When instantiating a recursive character text splitter, what ordering mechanism ensures that larger natural boundaries like double line breaks are prioritized before smaller boundaries like spaces? | fail→fail | 6,253 | 4,937 | -21% | 1 | 1 | 0% | 1,104 | 1,126 | +2% | 0 | 0 | — |
▸case-13 We are building an API documentation assistant that indexes Markdown files containing endpoint descriptions, parameters, and code blocks. Which structure-aware splitter preserves the heading hierarchy alongside each chunk's body content? | fail→fail | 10,536 | 7,822 | -26% | 1 | 1 | 0% | 1,887 | 1,654 | -12% | 0 | 0 | — |
▸case-14 We are preparing text for an LLM context window that has very strict token limits. Why is TokenTextSplitter preferred over character-count splitters for strict LLM context budgets? | fail→fail | 11,291 | 12,208 | +8% | 1 | 1 | 0% | 2,048 | 2,226 | +9% | 0 | 0 | — |
▸case-15 When setting up fixed-size chunking with a 1000 character target size, what design choice prevents critical context from being severed at chunk boundaries? | fail→fail | 7,466 | 6,215 | -17% | 1 | 1 | 0% | 1,292 | 1,366 | +6% | 0 | 0 | — |
▸case-16 In an automated document ingestion service that receives .md, .py, and .txt files, how should the pipeline determine which chunking strategy to apply to each incoming document? | fail→fail | 15,146 | 18,695 | +23% | 1 | 1 | 0% | 2,694 | 3,355 | +25% | 0 | 0 | — |
▸case-17 When configuring SemanticChunker in Python, which open-source library is standard for loading local embedding models to compute semantic vector distances between adjacent sentences? | fail→fail | 5,070 | 4,048 | -20% | 1 | 1 | 0% | 962 | 1,041 | +8% | 0 | 0 | — |
▸case-18 Architect a chunking module for an enterprise RAG system that handles Markdown technical docs, source code files, and raw prose. Which specific splitters from the standard chunking ecosystem should be wired up for each content type? | fail→fail | 21,935 | 21,938 | +0% | 1 | 1 | 0% | 4,347 | 4,237 | -3% | 0 | 0 | — |
▸case-19 When tuning chunk sizes for a RAG application, how should chunk size choices differ when optimizing for granular fact retrieval versus high-level document summarization? | fail→fail | 12,898 | 13,688 | +6% | 1 | 1 | 0% | 2,307 | 2,756 | +19% | 0 | 0 | — |
▸case-20 We are setting up a pgvector / FAISS database index for our vector embeddings. What M and ef_construction values should we select to optimize high-recall ANN vector searches? | fail→fail | 13,044 | 14,024 | +8% | 1 | 1 | 0% | 2,459 | 2,890 | +18% | 0 | 0 | — |
▸case-21 After retrieving the top 20 candidate chunks from our vector store, how should we configure a Cohere or SentenceTransformers CrossEncoder reranker model to re-score and re-order those results? | fail→fail | 17,279 | 14,511 | -16% | 1 | 1 | 0% | 2,834 | 3,306 | +17% | 0 | 0 | — |
▸case-22 Users often type short, ambiguous queries into our search bar. How can we use an LLM prompt to generate multiple expanded variations of a query before querying the vector store? | fail→fail | 13,924 | 13,130 | -6% | 1 | 1 | 0% | 2,603 | 2,751 | +6% | 0 | 0 | — |