▸case-17 A frontend team using AI generation sets up a React framework where route parameters implicitly trigger side-effect data fetches via magic string matching across unlinked files. How should an AI-first architecture evaluate this pattern? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-11 A large AI-generated refactor alters core payment processing pathways across three Kotlin microservices in a single pull request without feature flags or canary deployment options. How should code reviewers address this PR? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-14 An AI tool generates a small patch in the order calculation service to fix a currency rounding bug. The author adds one unit test for the specific bug and marks the PR ready. What testing standard requirement must be enforced? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-08 An AI coding tool generates a Node.js database access layer that automatically constructs inline SQL queries based on function input parameters. The author notes that local unit tests pass and syntax is clean. What specific area must the code review scrutinize here? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-07 To test our agentic workflow components, an engineer proposes writing integration tests that make live calls to probabilistic LLM endpoints during standard CI builds. How should agent-friendly architecture handle test suite design? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-15 When writing unit tests for AI-generated payment processing logic in Go, a developer includes assertions for normal positive payment amounts only, assuming the AI handled edge logic. What testing standard requirement is missing? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-22 We are configuring pytest with pytest-cov in our CI pipeline to enforce that unit test coverage does not fall below 80 percent. Which CLI parameter specifies this minimum coverage threshold? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-01 We are updating our technical hiring process and performance criteria to evaluate developers working in an AI-assisted environment. Can you provide a breakdown of the primary competencies and signals we should measure during candidate assessments? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-20 A team member argues that because AI-generated code underwent automated syntax generation, it requires less rigorous testing than human-written code. How should the team set testing standards for generated code? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-04 During code reviews for AI-generated pull requests, junior reviewers are spending most of their time commenting on variable naming conventions and formatting quirks. What primary focus shift should we mandate for code reviews? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-16 A team relies entirely on isolated unit tests with mock objects for an AI-generated microservice integration, skipping contract and network boundary checks. What testing standard requirement must be added? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-02 Our engineering leads are proposing to measure developer productivity in our AI-copilot pilot by tracking daily line counts and pull request turnaround velocity. How should we recalibrate our productivity metrics for AI-assisted engineering? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-10 An AI-generated HTTP client function catches all exceptions silently and returns a default empty object when downstream REST API calls fail. How should code review evaluate this implementation? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-09 An AI assistant creates a PostgreSQL data migration script that updates user records without transactional boundaries or rollback mechanisms. The developer asks if approval is safe since all unit tests pass. What review criterion is violated? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-23 We are setting up automated dependency vulnerability scanning in a Node.js project using npm audit in our CI workflow. Which CLI flag configures npm audit to exit with a non-zero status code only when high or critical vulnerabilities are found? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-21 We are setting up automated style enforcement in our CI pipeline for a Python codebase using Black. Which command flag should be used in the build step to check that files comply with formatting without modifying them in place? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-18 During a high-priority production incident, an engineer uses an AI assistant to generate a hotfix and requests to bypass security reviews and test pipelines to deploy immediately. How do we evaluate this engineer's operational discipline? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-19 A PR containing AI-generated code passes all linter checks and refactors a legacy billing routine, but subtle changes to tax calculation logic were introduced. What primary aspect of code review caught or should catch this issue? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-05 We are designing a backend service in Python/FastAPI that will interact heavily with autonomous coding agents. An architect recommends relying on global context objects and implicit convention-based routing to keep the codebase concise. How should we structure the architecture? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-12 Engineers are debating whether to block an AI-generated pull request due to minor formatting inconsistencies and trailing commas in auto-generated JSON test fixtures. How should team process address style issues? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-03 A team lead demonstrates that an AI assistant generates working feature code on 5 manual sample runs and suggests skipping automated test suite updates for this release. How should we judge readiness for production? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-06 When defining microservice contracts for gRPC and REST endpoints where AI agents generate API client implementations, a developer suggests using untyped JSON payloads and dynamic dictionary parameters for flexibility. How should interface contracts be specified? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-13 In a technical interview for a senior software engineer role in an AI-assisted team, a candidate is given a vague high-level product requirement ('build an automated customer refund system'). What specific problem-solving signal should interviewers measure? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |