▸case-01 Our team gets asked 'How do I submit an expense over $1,000 for client dinners?' in Slack at least twice a week. Right now I explain it orally: get pre-approval from Sarah, open a Jira ticket for accounting, and attach itemized receipts, though if it involves international travel or corporate card usage the steps diverge. Can you turn this tribal knowledge into an internal handbook page? Please format it with owner/status metadata, direct actionable steps, a realistic sample walkthrough, edge case branches, and a background context section. | fail→pass | 12,461 | 14,427 | +16% | 1 | 1 | 0% | 2,792 | 3,653 | +31% | 0 | 0 | — |
▸case-02 Engineers keep messaging me asking 'How do I get temporary root access to production databases during an incident?'. The current informal process involves opening a PagerDuty request, getting approval from on-call lead Dave, and using the Teleport CLI, but contractor accounts require additional security sign-off. Please draft a handbook document for this question. Include operational metadata up front, the step-by-step instructions, a end-to-end example scenario, exception handling, and background reasoning. | pass→pass | 16,430 | 11,100 | -32% | 1 | 1 | 0% | 3,048 | 3,191 | +5% | 0 | 0 | — |
▸case-03 I need to create a standardized wiki document for 'How do I request a hardware upgrade for my remote workstation?'. It's currently maintained informally by Alex in IT ops: employees submit an IT ticket with manager sign-off, though APAC team members follow a vendor drop-ship path. Could you turn this into a published handbook page structure containing document header metadata, primary procedural steps, a concrete example case, special exception branches, and general context at the end? | fail→pass | 15,221 | 9,959 | -35% | 1 | 1 | 0% | 2,532 | 2,861 | +13% | 0 | 0 | — |
▸case-04 We spend hours explaining 'How do I set up the corporate VPN on Linux?' in IT support chats. Users need to download the OpenVPN config, import it via NetworkManager, and authenticate using Okta TOTP, but ARM-based distributions require compiling the plugin from source. The team wants a traditional policy document titled 'SOP 402: Corporate Virtual Private Network Provisioning' starting with an overview of our Zero Trust network architecture, followed by steps, and ending with IT owner contacts. How should this handbook document be built? | fail→pass | 19,225 | 10,153 | -47% | 1 | 1 | 0% | 3,573 | 3,426 | -4% | 0 | 0 | — |
▸case-05 Our Slack admins keep getting pinged with 'How do I request a new public Slack channel?'. The process is filling out a workflow form, getting approval from a team lead, and setting the channel topic, but external partner channels require legal approval first. I want to write an essay introducing our communications policy and history of channel clutter before getting into the request steps. Please create the handbook page for this. | fail→pass | 13,240 | 10,449 | -21% | 1 | 1 | 0% | 2,351 | 2,922 | +24% | 0 | 0 | — |
▸case-06 DevOps receives daily questions asking 'How do I rotate my AWS IAM access key?'. The procedure involves creating a second key in AWS console, updating local credentials, testing, and deactivating the old key; root account keys cannot be self-rotated and must go through security. I'd like an overview-first wiki page titled 'AWS Access Key Lifecycle Management Policy' that explains IAM security best practices first. Please generate the proper handbook page. | fail→pass | 16,773 | 12,338 | -26% | 1 | 1 | 0% | 3,044 | 3,351 | +10% | 0 | 0 | — |
▸case-07 Procurement gets asked 'How do I onboard a new software vendor?' constantly. The workflow is submitting a vendor evaluation ticket, completing security questionnaire, and obtaining finance sign-off, but vendors handling PII require data privacy officer approval. Someone suggested creating a document opening with a history of vendor risks and procurement policies. Please construct the handbook page correctly. | fail→pass | 12,844 | 9,439 | -27% | 1 | 1 | 0% | 2,181 | 2,748 | +26% | 0 | 0 | — |
▸case-08 Offboarding HR asks 'How do I return my company laptop when leaving?'. Departing employees schedule a Courier pickup, pack the device in original box, and send tracking to HR, while remote employees outside North America use regional depot drop-off points. Create a handbook document, but make sure to open with company asset retention philosophy and hardware lifecycle background before giving instructions. | fail→pass | 9,161 | 9,858 | +8% | 1 | 1 | 0% | 1,566 | 2,869 | +83% | 0 | 0 | — |
▸case-09 Infrastructure receives repeated queries asking 'How do I register a new custom domain for a project?'. The process requires checking availability in Route53, getting marketing tag approval, and submitting a Cloud Infra request; top-level domains outside .com or .org need CISO authorization. Draft a handbook page, opening with domain governance philosophy. | fail→pass | 13,465 | 11,469 | -15% | 1 | 1 | 0% | 2,352 | 3,204 | +36% | 0 | 0 | — |
▸case-10 Finance staff are tired of answering 'How do I apply for a conference travel budget?'. The procedure is submitting a budget request form 30 days prior, securing manager sign-off, and logging estimated flight/hotel costs, but international travel requires VP approval. Draft the handbook page, ensuring context and history are kept at the bottom. | fail→pass | 7,775 | 9,404 | +21% | 1 | 1 | 0% | 1,402 | 2,758 | +97% | 0 | 0 | — |
▸case-11 Support agents repeatedly ask 'How do I issue a customer refund over $500?'. Agents verify purchase history in Stripe, request lead supervisor approval, and submit the refund transaction; enterprise accounts require account manager confirmation. Create a handbook page titled 'Refund Authorization Standard Operating Procedure' starting with company revenue retention policies. | fail→fail | 12,286 | 11,038 | -10% | 1 | 1 | 0% | 2,312 | 3,124 | +35% | 0 | 0 | — |
▸case-12 People Ops is inundated with 'How do I book a team offsite venue?'. The workflow involves choosing an approved vendor from the portal, getting travel team approval, and submitting the deposit invoice; offsites exceeding 20 attendees require facilities team notification. Please generate the handbook page. | fail→pass | 9,243 | 10,263 | +11% | 1 | 1 | 0% | 1,677 | 2,933 | +75% | 0 | 0 | — |
▸case-13 Engineers ask 'How do I schedule a production deployment window?'. The process is creating a release candidate tag in GitHub, opening a Change Advisory Board ticket, and notifying the incident channel; database schema migrations require DBA review. Create a handbook page formatted with task-first structure. | fail→pass | 7,968 | 8,820 | +11% | 1 | 1 | 0% | 1,514 | 2,917 | +93% | 0 | 0 | — |
▸case-14 Data team receives daily messages asking 'How do I request read access to Snowflake data warehouse?'. The process is requesting the snowflake-analyst role in Okta and getting manager approval; access to HIPAA-regulated tables requires compliance officer approval. Please produce a handbook page for this query. | fail→pass | 8,995 | 8,493 | -6% | 1 | 1 | 0% | 1,775 | 2,629 | +48% | 0 | 0 | — |
▸case-15 HR is asked 'How do I rollover unused PTO at year-end?'. Employees submit a rollover form in Workday by Dec 15 with manager approval; rollovers exceeding 5 days require HR director sign-off. Create a handbook page. | fail→pass | 8,750 | 10,052 | +15% | 1 | 1 | 0% | 1,722 | 2,894 | +68% | 0 | 0 | — |
▸case-16 On-call engineers are asked 'How do I page the primary on-call engineer during an outage?'. The step is executing the PagerDuty CLI command or triggering the Slack slash command; multi-region outages require escalating directly to executive incident management. Draft a handbook page. | fail→pass | 8,055 | 9,244 | +15% | 1 | 1 | 0% | 1,609 | 2,781 | +73% | 0 | 0 | — |
▸case-17 Compliance team receives repeated pings: 'How do I export customer data for a GDPR request?'. The steps are navigating to Admin Portal > Privacy, running the data extraction script, and uploading to secure S3 bucket; EU user data exports require DPO verification. Draft a handbook page. | fail→pass | 10,163 | 8,819 | -13% | 1 | 1 | 0% | 1,782 | 2,857 | +60% | 0 | 0 | — |
▸case-18 Engineering leads get asked 'How do I create a new repository under the corporate GitHub organization?'. The workflow is invoking the repo-bot Slack command, selecting standard language template, and setting branch protection rules; public open-source repos require open source board review. Build a handbook page. | fail→fail | 11,439 | 11,221 | -2% | 1 | 1 | 0% | 2,267 | 3,121 | +38% | 0 | 0 | — |
▸case-19 IT purchasing gets asked 'How do I request a license for JetBrains IDE?'. Users submit a ticket in Samanage with cost center code, and IT provisions key within 24 hours; contractor licenses require external worker agreement check. Create a handbook page. | fail→pass | 11,643 | 7,546 | -35% | 1 | 1 | 0% | 2,161 | 2,424 | +12% | 0 | 0 | — |
▸case-20 We had a 45-minute outage in our payment gateway service yesterday due to a database connection pool exhaustion. Can you help me draft a standard incident post-mortem report covering the incident timeline, root cause analysis, total financial impact, and action items to prevent recurrences? | pass→pass | 16,776 | 18,472 | +10% | 1 | 1 | 0% | 2,858 | 4,184 | +46% | 0 | 0 | — |
▸case-21 We need technical API reference documentation for our REST API endpoint POST /v1/billing/subscriptions. Please document the HTTP request headers, JSON request body parameters (plan_id, customer_id, auto_renew), expected 201 Created response payload, and standard HTTP error response codes (400, 401, 402). | pass→pass | 10,078 | 10,686 | +6% | 1 | 1 | 0% | 2,199 | 3,418 | +55% | 0 | 0 | — |
▸case-22 Here is a raw transcript of 15 questions asked in our #ask-devops Slack channel over the last month regarding Docker image builds, Kubernetes deployment manifests, and VPN connection drops. Please analyze this transcript and compile a raw list of top 5 most frequently asked questions along with a summary of recurring pain points. | fail→fail | 6,823 | 12,320 | +81% | 1 | 1 | 0% | 1,171 | 3,300 | +182% | 0 | 0 | — |