▸case-01 We are preparing our enterprise web application for a production security sign-off. Please generate a comprehensive security hardening audit document structured by standard domains (such as network topology, identity/access, API safety, and data protection). Format the response as a categorized markdown document with actionable verification items for each domain. | fail→pass | 49,876 | 41,668 | -16% | 1 | 1 | 0% | 7,144 | 3,674 | -49% | 0 | 0 | — |
▸case-02 I need to review our new cloud infrastructure and backend services against standard security hardening best practices. Can you provide a detailed security review report that outlines key verification checks for secrets management, supply chain security, incident response, and regulatory compliance? Present this as a checklist grouped by domain. | fail→pass | 29,016 | 11,003 | -62% | 1 | 1 | 0% | 4,064 | 2,648 | -35% | 0 | 0 | — |
▸case-03 Our team is building an API-first platform and needs a pre-deployment security review. Please construct an architectural security checklist covering network exposure, token authentication strategies, database encryption requirements, and gateway rate-limiting practices. Return the output as a clear domain-by-domain verification guide. | fail→fail | 29,449 | 18,659 | -37% | 1 | 1 | 0% | 4,224 | 3,362 | -20% | 0 | 0 | — |
▸case-04 Please write a complete Terraform configuration file in HCL to create an AWS VPC with two public subnets, two private subnets, an Internet Gateway, and a NAT Gateway. | pass→pass | 17,868 | 12,699 | -29% | 1 | 1 | 0% | 4,294 | 3,645 | -15% | 0 | 0 | — |
▸case-05 We have an ecommerce payment service that receives webhooks from Stripe and updates a PostgreSQL database. Please perform a threat modeling analysis using the STRIDE framework for this architecture. | pass→pass | 23,551 | 20,942 | -11% | 1 | 1 | 0% | 3,498 | 3,816 | +9% | 0 | 0 | — |
▸case-06 Please write a Python function using hashlib and hmac to sign incoming HTTP request bodies with SHA-256 for webhook verification. | pass→pass | 11,824 | 7,618 | -36% | 1 | 1 | 0% | 2,127 | 2,187 | +3% | 0 | 0 | — |
▸case-07 We are setting up authentication for our mobile web app. Developers suggested 1-hour access tokens and 30-day refresh tokens to minimize login prompts. Please audit our authentication token lifetime strategy and provide the required verification checklist. | fail→pass | 22,090 | 14,054 | -36% | 1 | 1 | 0% | 2,945 | 2,760 | -6% | 0 | 0 | — |
▸case-08 Our database team wants to rely on OS-level disk encryption and legacy TLS 1.0 for database connections in production. Provide a data security compliance checklist addressing transit, at-rest, and field-level encryption requirements. | fail→pass | 23,272 | 11,453 | -51% | 1 | 1 | 0% | 3,741 | 2,647 | -29% | 0 | 0 | — |
▸case-09 Our build system currently injects database passwords as plain-text environment variables in Kubernetes pod specs. Provide a secrets management security checklist detailing the required infrastructure tools and rotation practices. | pass→pass | 20,720 | 15,092 | -27% | 1 | 1 | 0% | 3,092 | 2,801 | -9% | 0 | 0 | — |
▸case-16 Our REST API relies entirely on frontend JavaScript checks for input validation and has no rate limits on login endpoints. Provide an API security checklist covering gateway rate-limiting and backend injection defenses. | pass→pass | 17,396 | 13,219 | -24% | 1 | 1 | 0% | 2,665 | 2,500 | -6% | 0 | 0 | — |
▸case-10 Our software delivery team relies solely on manual code reviews before merging pull requests and releases container images without scanning. Create a supply chain and dependency security checklist for our CI/CD pipeline. | pass→pass | 20,960 | 15,403 | -27% | 1 | 1 | 0% | 3,160 | 2,786 | -12% | 0 | 0 | — |
▸case-11 Our legal team asked what incident response timelines and audit requirements should be mandated in our security guidelines. Provide an incident response checklist including breach notification timeframes. | fail→pass | 20,267 | 15,580 | -23% | 1 | 1 | 0% | 2,961 | 2,726 | -8% | 0 | 0 | — |
▸case-12 Our cloud team proposed putting microservices in public subnets with open ingress ports to simplify inter-service debugging. Create a network security checklist specifying VPC isolation, gateway rules, and inter-region communication. | fail→fail | 21,520 | 13,782 | -36% | 1 | 1 | 0% | 3,075 | 2,672 | -13% | 0 | 0 | — |
▸case-13 To resolve browser origin errors quickly, our front-end team set Access-Control-Allow-Origin to an unrestricted wildcard across all production endpoints. Provide an API security checklist covering CORS configuration and required security headers. | pass→pass | 20,464 | 17,786 | -13% | 1 | 1 | 0% | 2,820 | 2,472 | -12% | 0 | 0 | — |
▸case-14 We are expanding our platform to handle payment card numbers and healthcare records. Reviewers suggested storing credit card PANs in internal logs for troubleshooting. Provide a compliance checklist covering health and payment data standards. | pass→pass | 19,611 | 17,271 | -12% | 1 | 1 | 0% | 2,907 | 3,013 | +4% | 0 | 0 | — |
▸case-15 Our developers plan to store admin database passwords and full user profiles inside JWT tokens so backend services do not need to look them up. Provide an identity and authorization security checklist. | fail→pass | 20,195 | 13,792 | -32% | 1 | 1 | 0% | 2,759 | 2,525 | -8% | 0 | 0 | — |
▸case-17 Our debugging setup outputs full HTTP request bodies—including user passwords and credit card numbers—to centralized logging tools. Provide a data security checklist regarding logging practices. | pass→pass | 22,115 | 14,551 | -34% | 1 | 1 | 0% | 3,005 | 2,665 | -11% | 0 | 0 | — |
▸case-18 Engineers are embedding static IAM user access keys in application config files for inter-service API calls. Provide a secrets and identity management checklist covering service-to-service authentication. | pass→pass | 17,247 | 16,306 | -5% | 1 | 1 | 0% | 2,628 | 3,273 | +25% | 0 | 0 | — |
▸case-19 Management wants to skip WAF and DDoS protection to save on cloud operational costs. Create a network perimeter security checklist covering WAF and DDoS requirements. | fail→pass | 18,394 | 12,056 | -34% | 1 | 1 | 0% | 2,740 | 2,484 | -9% | 0 | 0 | — |
▸case-20 Our deployment pipeline uses floating version tags for all node packages and container bases. Provide a supply chain security checklist addressing dependency version management. | pass→pass | 17,818 | 11,611 | -35% | 1 | 1 | 0% | 2,631 | 2,679 | +2% | 0 | 0 | — |
▸case-21 Our authentication service issues stateless JWTs but has no mechanism to revoke compromised user sessions before token expiration. Provide an auth checklist covering token revocation mechanisms. | pass→pass | 16,092 | 14,106 | -12% | 1 | 1 | 0% | 2,580 | 2,916 | +13% | 0 | 0 | — |
▸case-22 Our database backup job writes unencrypted SQL dumps directly to standard cloud storage buckets. Provide a data protection security checklist for database backups. | pass→pass | 17,690 | 13,058 | -26% | 1 | 1 | 0% | 2,746 | 2,663 | -3% | 0 | 0 | — |
▸case-23 System administrators currently log into the management console using single-factor password authentication. Provide an identity security checklist for admin account access. | pass→pass | 16,069 | 12,577 | -22% | 1 | 1 | 0% | 2,332 | 2,277 | -2% | 0 | 0 | — |
▸case-24 Our team only plans to test security after a security breach occurs. Provide an incident response security checklist covering proactive security testing and monitoring. | pass→pass | 17,255 | 14,817 | -14% | 1 | 1 | 0% | 2,642 | 2,612 | -1% | 0 | 0 | — |
▸case-25 Our web application accepts raw user HTML without encoding and stores session tokens in plain cookies without SameSite flags. Provide an API security checklist covering XSS and CSRF mitigations. | pass→pass | 20,889 | 13,618 | -35% | 1 | 1 | 0% | 2,954 | 2,609 | -12% | 0 | 0 | — |
▸case-26 We are preparing for a SOC 2 Type II audit report for our enterprise clients. Provide a compliance checklist outlining key audit evidence requirements. | pass→pass | 20,505 | 17,855 | -13% | 1 | 1 | 0% | 3,054 | 3,342 | +9% | 0 | 0 | — |
▸case-27 Our default cloud security group permits inbound traffic on all ports from all IP addresses to allow microservices to communicate freely. Provide a network security checklist addressing security group rules. | pass→pass | 15,971 | 11,923 | -25% | 1 | 1 | 0% | 2,612 | 2,364 | -9% | 0 | 0 | — |
▸case-28 Our portal allows third-party login but uses custom proprietary HTTP header parsing instead of standardized identity protocols. Provide an authentication checklist for third-party auth integrations. | fail→pass | 18,979 | 15,578 | -18% | 1 | 1 | 0% | 2,890 | 3,233 | +12% | 0 | 0 | — |