▸case-16 We have an upcoming release that combines a major PostgreSQL version upgrade, a Kubernetes cluster version upgrade, a total rewrite of our NGINX ingress routing rules, and secrets rotation for all microservices into a single weekend deployment. How should this operational change be planned? | fail→pass | 31,777 | 19,582 | -38% | 1 | 1 | 0% | 3,493 | 3,518 | +1% | 0 | 0 | — |
▸case-02 We are updating our AWS security groups and IAM permissions for the payment microservice. Can you guide me through verifying these IaC changes safely, ensuring we inspect the diff for risks and list any commands that haven't been run yet before we touch any live resources? | fail→fail | 22,196 | 23,907 | +8% | 1 | 1 | 0% | 2,375 | 2,961 | +25% | 0 | 0 | — |
▸case-03 I have a pull request ready that modifies our Helm values and updates the database migration script for our billing service. Walk me through running our validation checks, explicitly calling out the target environment being checked and separating proposed actions from completed verifications. | fail→fail | 24,598 | 26,572 | +8% | 1 | 1 | 0% | 2,709 | 3,495 | +29% | 0 | 0 | — |
▸case-04 We are adding a new AWS S3 bucket and CloudFront distribution using Terraform in our media-service repo. A team member suggested running Checkov policy scanning right after `terraform fmt` to catch policy violations before running `terraform plan` or rendering local diffs. Can you provide the correct sequence for validating and verifying these IaC changes before applying? | pass→pass | 22,201 | 23,033 | +4% | 1 | 1 | 0% | 2,464 | 2,869 | +16% | 0 | 0 | — |
▸case-01 I need to update our staging and production Kubernetes ingress configurations and scale up the Postgres RDS instance. Please review my configuration changes, run through the necessary pre-flight checks and preview steps, and provide a report that separates what is ready to apply, what needs approval, and how we will reverse the changes if issues occur. | fail→fail | 23,542 | 16,662 | -29% | 1 | 1 | 0% | 2,343 | 866 | -63% | 0 | 0 | — |
▸case-05 Our GitLab CI pipeline for the payment-gateway infrastructure currently runs `terraform plan` on merge to `main`. To speed up deployments, a developer proposed adding `terraform apply -auto-approve` to the same pipeline job if `terraform plan` exits with code 0. Evaluate this proposed pipeline change and recommend the proper workflow. | pass→pass | 23,647 | 17,884 | -24% | 1 | 1 | 0% | 2,822 | 3,325 | +18% | 0 | 0 | — |
▸case-06 To ensure we always use the latest features and security fixes when building our Docker container images and running GitHub Actions in our microservice deployment pipeline, a developer set all GitHub Action versions to `@main` and base images to `golang:latest`. How should we manage these pipeline dependencies? | pass→pass | 22,765 | 23,059 | +1% | 1 | 1 | 0% | 2,489 | 3,007 | +21% | 0 | 0 | — |
▸case-07 An engineer created a local helper script `deploy-staging.sh` for our Kubernetes services that embeds the AWS access key `AKIAEXAMPLE` and secret key directly in the script header so developers do not need to log into AWS SSO each time. How should credentials and identities be handled for infrastructure management tasks? | pass→pass | 15,193 | 18,930 | +25% | 1 | 1 | 0% | 2,277 | 2,653 | +17% | 0 | 0 | — |
▸case-08 During a production hotfix for our authentication service's Terraform configuration, OPA Gatekeeper policy tests failed because the new IAM role includes wildcard resource access `arn:aws:s3:::*`. The developer wants to set `failure_threshold: ignore` in the CI pipeline to unblock the build. How should this policy failure be addressed? | pass→pass | 21,274 | 18,995 | -11% | 1 | 1 | 0% | 2,081 | 2,731 | +31% | 0 | 0 | — |
▸case-09 A developer wrote an automation script that runs `kubectl apply -f manifests/` using whichever Kubernetes cluster context is currently active in the user's local `~/.kube/config`. Review this execution approach and explain how target environment verification should be structured. | fail→pass | 22,562 | 21,406 | -5% | 1 | 1 | 0% | 3,151 | 3,570 | +13% | 0 | 0 | — |
▸case-10 We are updating our user database schema in PostgreSQL and cutting over traffic to a new API cluster using AWS Route 53. The migration plan proposes dropping the old database tables and deleting the old Route 53 routing policy immediately after traffic reaches 100% on the new cluster. Evaluate this cutover plan. | pass→pass | 21,636 | 25,203 | +16% | 1 | 1 | 0% | 2,609 | 3,050 | +17% | 0 | 0 | — |
▸case-11 I just finished running local linting and syntax validation for our ArgoCD application manifests in the billing repository. Please generate a summary status report for the team that outlines what we checked, what remains to be done, and what commands were omitted during testing. | pass→pass | 9,968 | 20,809 | +109% | 1 | 1 | 0% | 1,613 | 2,490 | +54% | 0 | 0 | — |
▸case-12 An engineer ran `terragrunt validate` on our infrastructure repository and reported: 'Validated successfully in dev cloud account.' Review whether this reporting statement provides sufficient detail for an operational audit log. | pass→pass | 20,264 | 9,269 | -54% | 1 | 1 | 0% | 2,688 | 1,906 | -29% | 0 | 0 | — |
▸case-13 A cleanup script designed to free up cloud resources is scheduled to run `terraform destroy` against our shared staging Redis cluster and RDS database every Friday evening automatically. Assess this automated cleanup strategy. | fail→pass | 23,019 | 21,592 | -6% | 1 | 1 | 0% | 2,348 | 2,821 | +20% | 0 | 0 | — |
▸case-14 We are preparing a Helm release for our notification microservice. A team member wants to run `helm diff upgrade` directly against our production Kubernetes cluster before running `helm lint` or `helm template`. What is the required sequence of validation steps? | pass→pass | 20,308 | 17,862 | -12% | 1 | 1 | 0% | 2,133 | 2,429 | +14% | 0 | 0 | — |
▸case-15 Our deployment pipeline for the search microservice executes `helm upgrade` and considers the deployment complete as soon as the Kubernetes pod status reports `Running`. Is this deployment completion criteria sufficient? | fail→pass | 12,523 | 19,182 | +53% | 1 | 1 | 0% | 1,946 | 2,415 | +24% | 0 | 0 | — |
▸case-17 Please review this Git diff for our Ansible deployment script: `+ git clone https://github.com/example/deploy-tools.git /opt/tools`. The developer claims this is ready for production merge. What risk should be identified in this code review? | pass→pass | 16,673 | 10,619 | -36% | 1 | 1 | 0% | 1,800 | 1,888 | +5% | 0 | 0 | — |
▸case-18 A shell script executes `terraform apply` directly without querying AWS STS to check `aws sts get-caller-identity`. The author assumes the developer's terminal environment variables are always pointed to the staging account. What safeguard should be added to this script? | pass→pass | 16,622 | 16,035 | -4% | 1 | 1 | 0% | 1,745 | 2,084 | +19% | 0 | 0 | — |
▸case-19 I need you to write a Python FastAPI endpoint that receives a JSON payload containing `user_id` and `amount`, validates the input fields using Pydantic, and calls an external payment gateway API. | pass→pass | 15,072 | 21,387 | +42% | 1 | 1 | 0% | 3,128 | 3,487 | +11% | 0 | 0 | — |
▸case-20 Write a React TypeScript functional component that displays an interactive data grid with sorting, pagination, and filter dropdowns for an admin dashboard. | pass→pass | 34,000 | 34,496 | +1% | 1 | 1 | 0% | 6,514 | 7,835 | +20% | 0 | 0 | — |
▸case-21 Draft an end-user help article explaining how non-technical users can reset their two-factor authentication from their account settings menu in our web application. | pass→pass | 16,599 | 16,635 | +0% | 1 | 1 | 0% | 1,957 | 2,123 | +8% | 0 | 0 | — |
▸case-22 Write an optimized SQL query for PostgreSQL that calculates the 30-day rolling average revenue per user (ARPU) from our `invoices` and `users` tables. | pass→pass | 26,397 | 16,369 | -38% | 1 | 1 | 0% | 3,779 | 3,503 | -7% | 0 | 0 | — |