▸case-10 In a multi-agent diagnostic chain where Agent A (Triage) passes findings to Agent B (Remediation), should Agent B receive the entire raw conversation transcript of 50 turns, or a structured handoff state schema with synthesized context? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-04 We are building a simple single-turn sentiment analysis service where input text is evaluated for positive, neutral, or negative sentiment. Should we implement a dynamic context retrieval engine with vector database lookups for every incoming request? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-02 Our AI assistant frequently hits token limits when pulling large technical documents during complex troubleshooting queries. Could you produce a token budget management guide for context window optimization? The output should include an actionable step-by-step strategy for chunking, relevance pruning, context compression, and concrete metrics to verify context quality. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-03 I am designing a persistent memory system for a long-running conversational agent that needs both episodic memory for interaction history and semantic memory for user preferences. Please draft a memory architecture design document outlining storage structures, consolidation mechanisms, hybrid search retrieval workflows, and testing criteria to validate memory retrieval accuracy. | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-09 We are merging customer interactions across email, chat, and phone channels into a unified graph store. How should we handle entity resolution when 'J. Doe' in chat might be 'John Doe' in email? | pass→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-12 When an AI assistant learns a user's persistent preference during a single chat session, should that preference remain only in the episodic chat history or be consolidated into long-term semantic memory? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-01 We are building an enterprise workflow where several specialized AI agents handle different stages of customer support tickets. I need a clear context orchestration plan that defines how agent-to-agent state handoffs should be structured. Please provide an architectural proposal detailing the state schema for memory transfer, the dynamic context assembly process, and validation steps to ensure data isn't lost during agent transitions. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-15 When structuring an LLM prompt payload with dynamic components (system instructions, user query, retrieved RAG chunks, chat history), how should token budget allocation be prioritized? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-22 When setting up chunking for 100-page dense technical manuals in a vector database, an engineer suggests using ultra-small 50-token chunks with no parent document expansion to maximize embedding specificity. Why is this approach problematic for complex reasoning, and what retrieval strategy solves it? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-05 For a fixed text classification task with four predefined classes, should we implement a dynamic RAG retrieval pipeline to pull example documents per request or embed fixed few-shot examples directly in the system prompt? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-16 For multi-hop question answering over connected technical dependencies, should we rely solely on single-step vector embedding lookup or perform graph traversal query pathing? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-08 When splitting 50-page PDF technical manuals for dense retrieval in a technical support bot, should we chunk by raw token length of 2000 tokens with 0 overlap, or use sliding semantic windows with a 10-20% overlap between chunks? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-17 In a regulated medical AI assistant, we must track exactly what context chunks were provided to the model for every generated response. How should context versioning and logging be implemented? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-07 We are building a vector search pipeline using BM25 keyword matching alongside dense vector similarity. Should we combine their scores using simple raw score addition, or should we use Reciprocal Rank Fusion (RRF) to merge ranks across both sparse and dense retrieval systems? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-18 In a multi-tenant SaaS application sharing a single vector database index, how should we prevent tenant context leakage during similarity queries? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-13 Our enterprise knowledge base updates every hour, but users complain the RAG assistant answers with stale documentation. How can we ensure dynamic context freshness in vector retrieval? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-19 For real-time multi-turn conversational agents, how should working memory be updated as new turns arrive? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-20 To improve precision in top-10 retrieved chunks from a vector search of 1,000,000 documents, should we feed vector top-50 results to a cross-encoder re-ranker model? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-11 Our agent context window is full after 10 turns of code debugging. Should we blindly drop the middle 5 messages or apply hierarchical context summarization while preserving critical code snippets? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-06 We have an automated pipeline that sends single short emails (under 200 words) to an LLM to generate a quick bulleted summary. Should we build an episodic context memory store and sliding window summarization pipeline for these requests? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-14 An AI agent needs to call a SQL execution tool based on user prompt context. Should the agent pass unstructured user prompt text directly as the database query parameter, or extract sanitized parameters into a structured JSON payload first? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-21 How can we automatically evaluate whether retrieved RAG context chunks are actually relevant to a user query before passing them to the generator model? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |