▸case-01 I'm designing an automated customer support system using Google's Agent Development Kit that handles order lookups via internal APIs and escalates complex billing issues to human staff. Could you help me structure this project? Please provide a recommended agent architecture layout, a step-by-step implementation plan with a folder scaffold, and a testing checklist for both local development and CI pipeline verification. | fail→fail | 27,466 | 25,784 | -6% | 1 | 1 | 0% | 5,303 | 4,741 | -11% | 0 | 0 | — |
▸case-02 We are building a document processing assistant using Google ADK that ingests PDFs, queries BigQuery, and formats summaries. We need help deciding whether a unified agent or an orchestrated multi-agent pattern fits best. Please evaluate the architectural options, outline the project scaffold, and supply a command checklist for local smoke testing and post-deployment verification. | fail→fail | 23,533 | 17,686 | -25% | 1 | 1 | 0% | 4,962 | 3,997 | -19% | 0 | 0 | — |
▸case-03 I have a Python prototype for an IT helpdesk bot that interacts with ServiceNow APIs, and I need to standardize it for production on Vertex AI Agent Engine using Google ADK patterns. Can you guide me on setting up clear tool error contracts, a clean scaffold structure, and an operational validation plan covering regression tests and deployment health checks? | fail→fail | 21,313 | 26,390 | +24% | 1 | 1 | 0% | 4,972 | 5,088 | +2% | 0 | 0 | — |
▸case-04 We need to provision a GKE cluster on Google Cloud using Terraform HCL, including node pools and VPC subnetting. Please provide a clean terraform file structure and resource configuration blocks for google_container_cluster. | pass→pass | 14,880 | 20,439 | +37% | 1 | 1 | 0% | 3,626 | 4,263 | +18% | 0 | 0 | — |
▸case-05 I want to launch a custom PyTorch hyperparameter tuning job on Vertex AI using gcloud CLI. Can you show me how to format the hyperparameterSpec YAML configuration and the gcloud ai hp-tuning-jobs create command? | pass→pass | 17,604 | 16,825 | -4% | 1 | 1 | 0% | 2,896 | 3,240 | +12% | 0 | 0 | — |
▸case-06 I am writing a standalone local Python script with LangChain that uses an SQLite tool to fetch inventory data. Can you give me a minimal Python example using initialize_agent or LangChain create_react_agent with sqlite3? | pass→pass | 10,881 | 12,467 | +15% | 1 | 1 | 0% | 2,484 | 3,539 | +42% | 0 | 0 | — |
▸case-07 We are creating a financial advisory assistant on Google Cloud that combines market sentiment analysis, portfolio risk calculation, and trade execution. A developer suggested putting all prompt logic and API calls into one single python script file app.py. Is a single agent or multi-agent orchestrator better here, and what directory layout should we use? | pass→pass | 20,480 | 21,341 | +4% | 1 | 1 | 0% | 3,140 | 3,984 | +27% | 0 | 0 | — |
▸case-08 We are wiring an e-commerce stock lookup tool to a Google Cloud agent. Our backend API often fails with HTTP 503 or return malformed JSON when warehouse DBs are under heavy load. The dev team plans to just pass raw Python dict responses straight back to the model without defining error schemas. How should we formalize the tool interface contract and handle failures? | fail→pass | 19,003 | 28,172 | +48% | 1 | 1 | 0% | 3,018 | 4,429 | +47% | 0 | 0 | — |
▸case-09 We are building a healthcare symptom checker agent for deployment on Vertex AI Agent Engine. The clinical team wants to hardcode the database secret key directly inside tools/patient_db.py to simplify local development. What prerequisites and secret management patterns must we enforce before writing agent logic? | fail→fail | 22,603 | 21,739 | -4% | 1 | 1 | 0% | 2,775 | 3,389 | +22% | 0 | 0 | — |
▸case-10 Our team is asked to build a real estate property search bot using Google's Agent Development Kit patterns. The project manager wants us to build everything at once in a single release sprint. How should we structure an implementation plan with clear incremental milestones? | fail→pass | 24,712 | 26,279 | +6% | 1 | 1 | 0% | 2,924 | 3,816 | +31% | 0 | 0 | — |
▸case-11 Our logistics dispatch agent uses a third-party mapping API requiring an API key, and calls Google Cloud Spanner for route data. A developer suggested reading API keys from a plain unencrypted .env file checked into Git. How should tool secrets be configured for both local development and Vertex AI Agent Engine? | pass→pass | 15,881 | 19,609 | +23% | 1 | 1 | 0% | 2,790 | 3,967 | +42% | 0 | 0 | — |
▸case-12 We built a travel reservation agent on Google Cloud. The team is currently testing by manually typing 'Book a flight to Tokyo' into a CLI terminal. How should we implement automated validation, including regression tests and smoke prompts? | fail→pass | 24,524 | 19,425 | -21% | 1 | 1 | 0% | 3,226 | 4,117 | +28% | 0 | 0 | — |
▸case-13 We have an incident triage agent built with Google ADK patterns. The QA engineer asked for a bulleted list of high-level testing concepts for CI. We need exact executable shell commands to run in our CI/CD pipeline for local validation and linting. What commands should be specified? | pass→pass | 17,525 | 10,163 | -42% | 1 | 1 | 0% | 2,086 | 2,202 | +6% | 0 | 0 | — |
▸case-14 Our legal document analyzer agent calls a third-party OCR API. When the PDF is password protected, the OCR API throws a custom HTTP 422 error. Junior developers want the agent to crash with an unhandled exception so they can inspect stack traces. How should this error contract be handled in tool definitions? | fail→pass | 18,536 | 19,707 | +6% | 1 | 1 | 0% | 2,313 | 3,076 | +33% | 0 | 0 | — |
▸case-15 For a personalized retail product recommendation agent built with Google ADK, we need to establish validation protocols. A tester suggested only checking whether the server returns HTTP 200. What smoke prompts and test inputs should be included to validate reasoning? | pass→pass | 19,460 | 28,899 | +49% | 1 | 1 | 0% | 3,222 | 4,701 | +46% | 0 | 0 | — |
▸case-16 We are deploying a telecom network diagnostic agent to Vertex AI Agent Engine. The team plan is to deploy directly to production without local validation or post-deployment health checks. What operational validation steps and health checks are required? | fail→pass | 18,077 | 29,230 | +62% | 1 | 1 | 0% | 2,439 | 4,600 | +89% | 0 | 0 | — |
▸case-17 We need an HR onboarding assistant that handles policy Q&A, account provisioning via IT APIs, and badge request submissions. Should we construct a single monolithic prompt or a multi-agent orchestrator pattern? Please explain the architectural tradeoffs. | pass→pass | 16,504 | 16,905 | +2% | 1 | 1 | 0% | 2,685 | 3,693 | +38% | 0 | 0 | — |
▸case-18 We are starting a SecOps alert triage assistant using Google ADK patterns. A developer proposed creating a flat directory with all code in one folder. What standard folder scaffold layout should be created for agents, tools, config, and tests? | pass→pass | 16,050 | 12,348 | -23% | 1 | 1 | 0% | 2,835 | 2,619 | -8% | 0 | 0 | — |
▸case-19 Our vehicle fleet tracking agent has been deployed to Google Cloud Agent Engine. How should we verify that the deployed live service endpoint is functioning properly in production? | fail→pass | 16,253 | 25,852 | +59% | 1 | 1 | 0% | 2,623 | 3,616 | +38% | 0 | 0 | — |
▸case-20 We are building an internal code refactoring agent that parses Git diffs and suggests style fixes. The team wants a execution plan. Should we write all agent code, tools, and tests in one single step, or break it into incremental milestones? | fail→fail | 18,710 | 19,124 | +2% | 1 | 1 | 0% | 2,194 | 2,982 | +36% | 0 | 0 | — |
▸case-21 A marketing copy agent uses Google Cloud Vertex AI and needs credentials for Mailchimp and Twitter/X APIs. A developer wants to hardcode token strings directly into tools/social_media.py. How must credentials be handled? | pass→pass | 12,526 | 15,587 | +24% | 1 | 1 | 0% | 1,839 | 2,894 | +57% | 0 | 0 | — |
▸case-22 We are setting up GitHub Actions CI for a hotel concierge booking agent. The lead developer wrote pseudo-code descriptions in the CI document like 'Check that code compiles'. What specific CLI commands belong in the CI test checklist? | pass→pass | 23,048 | 21,293 | -8% | 1 | 1 | 0% | 2,794 | 2,846 | +2% | 0 | 0 | — |
▸case-23 We are developing an insurance claims processing agent. One legacy internal doc mentions using an outdated pattern, while 000-docs/6767-* specifies current plugin standards. Which standard takes precedence when conflicting guidance occurs? | fail→fail | 3,180 | 8,598 | +170% | 1 | 1 | 0% | 505 | 982 | +94% | 0 | 0 | — |