▸case-06 A commit in git repo `backend-core` has summary 'fix: fix bugs' and empty body. Team guidelines follow Conventional Commits standard. Is this commit message adequate for git log history auditing? | pass→pass | 9,890 | 8,216 | -17% | 1 | 1 | 0% | 1,614 | 1,396 | -14% | 0 | 0 | — |
▸case-02 We have a Jira ticket target PROJ-882 with the description: 'Fix the bug in the login pipeline where users report issues during authentication.' Developer wants to close the refinement review saying it is ready for sprint planning. Evaluate whether this description satisfies standard Jira backlog refinement requirements or needs further specification. | pass→pass | 11,058 | 10,741 | -3% | 1 | 1 | 0% | 1,709 | 1,814 | +6% | 0 | 0 | — |
▸case-03 Pull Request #104 in repo 'acme/auth-service' has the body text: 'Updated various files to improve security and clean up code.' A automated CI linter is checking if this description meets GitHub PR documentation standards. Should this PR body be approved as complete, or flagged as a placeholder? | pass→pass | 6,382 | 3,611 | -43% | 1 | 1 | 0% | 1,052 | 724 | -31% | 0 | 0 | — |
▸case-19 A code review comment on line 45 reads: 'Fix this line.' The author asks if this review feedback is actionable. | pass→pass | 5,935 | 4,960 | -16% | 1 | 1 | 0% | 1,140 | 1,002 | -12% | 0 | 0 | — |
▸case-04 User story US-204 states: 'As a user, I want the system to be fast so that I have a good experience.' The product manager wants to push this directly to dev estimation. Evaluate this story against the INVEST framework. | pass→pass | 9,162 | 11,845 | +29% | 1 | 1 | 0% | 1,862 | 1,840 | -1% | 0 | 0 | — |
▸case-05 In an OpenAPI 3.0 document for `/api/v1/payments`, the `description` field for `POST /charge` is set to 'Handles charges.' API reviewers are considering passing this for public developer documentation release. Is this description compliant with OpenAPI quality guidelines? | fail→fail | 8,528 | 8,591 | +1% | 1 | 1 | 0% | 1,606 | 1,630 | +1% | 0 | 0 | — |
▸case-01 Can you inspect this task description to see if it is overly vague or just acting as a filler placeholder, and return a report detailing what specific information needs to be added? | fail→pass | 4,516 | 6,867 | +52% | 1 | 1 | 0% | 734 | 1,242 | +69% | 0 | 0 | — |
▸case-07 In issue tracker Bugzilla #4920, the bug report says 'App crashes on checkout screen sometimes.' The QA lead asks if this ticket can be assigned directly to an engineer for root cause fix. | fail→pass | 10,229 | 6,060 | -41% | 1 | 1 | 0% | 1,498 | 971 | -35% | 0 | 0 | — |
▸case-08 In a technical spec document for an OAuth2 server, Section 3.2 states: 'The system should process tokens properly.' Is this statement enforceable in an RFC 2119 conformant engineering specification? | pass→pass | 11,426 | 8,221 | -28% | 1 | 1 | 0% | 1,686 | 1,359 | -19% | 0 | 0 | — |
▸case-09 In an IEEE 1016 software design document, the database component description states: 'Stores application data efficiently.' The lead architect asks if this architectural description is sufficient for system review. | pass→pass | 10,476 | 10,046 | -4% | 1 | 1 | 0% | 1,829 | 1,481 | -19% | 0 | 0 | — |
▸case-10 A draft release note for version 2.4.0 states: 'Miscellaneous bug fixes and performance improvements.' The release manager wants to publish this to external enterprise customers. Evaluate this release note against software changelog best practices. | pass→pass | 13,645 | 11,515 | -16% | 1 | 1 | 0% | 2,064 | 1,951 | -5% | 0 | 0 | — |
▸case-11 A draft CVE vulnerability report for internal advisory SEC-2026-01 reads: 'A vulnerability exists in the auth module allowing bad things to happen.' Does this meet CVE project description requirements? | pass→pass | 8,895 | 13,215 | +49% | 1 | 1 | 0% | 1,483 | 1,228 | -17% | 0 | 0 | — |
▸case-12 In a Terraform module `modules/aws-s3`, a variable definition has: `variable "bucket_name" { description = "The bucket name" }`. Code reviewer wants to enforce tf-docs best practices. | pass→pass | 8,485 | 4,804 | -43% | 1 | 1 | 0% | 1,385 | 650 | -53% | 0 | 0 | — |
▸case-13 A task ticket SCRUM-412 in Jira states: 'Refactor database code.' The Scrum master asks if this task meets the Definition of Ready for sprint planning. | pass→pass | 9,146 | 8,007 | -12% | 1 | 1 | 0% | 1,407 | 1,481 | +5% | 0 | 0 | — |
▸case-14 An incident post-mortem action item states: 'Make monitoring better so this doesn't happen again.' The Reliability Director asks if this action item is actionable. | pass→pass | 9,641 | 8,832 | -8% | 1 | 1 | 0% | 1,506 | 1,488 | -1% | 0 | 0 | — |
▸case-15 In a GraphQL schema file, field `userStatus: String` has the description comment `"""The status"""`. Schema reviewer checks if this meets GraphQL schema documentation guidelines. | pass→pass | 9,274 | 6,087 | -34% | 1 | 1 | 0% | 1,444 | 958 | -34% | 0 | 0 | — |
▸case-16 A user story ticket includes the acceptance criterion: 'Feature works correctly and is tested.' Does this acceptance criterion meet SMART guidelines? | pass→pass | 8,445 | 8,932 | +6% | 1 | 1 | 0% | 1,423 | 1,456 | +2% | 0 | 0 | — |
▸case-17 A technical debt ticket DEBT-99 states: 'Clean up technical debt in the frontend repo.' The engineering manager asks whether to prioritize this in the upcoming quarter. | fail→pass | 11,567 | 8,757 | -24% | 1 | 1 | 0% | 1,712 | 1,566 | -9% | 0 | 0 | — |
▸case-18 In a Helm chart `values.yaml`, a parameter comment reads `# Configures configuration settings.` Helm repository reviewer evaluates this for artifact hub publishing. | pass→pass | 7,588 | 9,373 | +24% | 1 | 1 | 0% | 1,155 | 1,392 | +21% | 0 | 0 | — |
▸case-20 Write a Jira Query Language (JQL) query to find all issues in project 'PROJ' updated in the last 7 days with status 'In Progress'. | pass→pass | 2,460 | 2,107 | -14% | 1 | 1 | 0% | 433 | 382 | -12% | 0 | 0 | — |
▸case-21 Format the following raw text into a standard Markdown table with columns 'Feature', 'Status', and 'Owner': Auth (Done, Alice), Billing (In Progress, Bob). | pass→pass | 1,429 | 1,566 | +10% | 1 | 1 | 0% | 259 | 308 | +19% | 0 | 0 | — |
▸case-22 What is the standard git branch naming syntax for a feature branch addressing issue PROJ-123 in GitFlow? | pass→pass | 6,157 | 2,927 | -52% | 1 | 1 | 0% | 1,104 | 586 | -47% | 0 | 0 | — |