▸case-10 Our CI pipeline runs automated desktop accessibility scans on static UI mockups, but skips keyboard interaction tests and focus trapping checks to save time in build steps. Assess our test coverage plan. | pass→pass | 15,557 | 16,733 | +8% | 1 | 1 | 0% | 2,493 | 2,610 | +5% | 0 | 0 | — |
▸case-15 An MCP accessibility server relies on downloading 2GB of uncompressed node modules on every tool invocation. Evaluate this architecture for developer tooling efficiency. | pass→pass | 16,462 | 14,747 | -10% | 1 | 1 | 0% | 2,400 | 2,281 | -5% | 0 | 0 | — |
▸case-07 Our project contains a complex mix of desktop GUI components and scanner scripts. The lead developer asked to run a focused subagent review on the wxPython front-end using standard Codex tools. How should subagent invocation be organized for this review? | pass→fail | 14,276 | 8,488 | -41% | 1 | 1 | 0% | 2,387 | 1,430 | -40% | 0 | 0 | — |
▸case-08 For a C++ desktop GUI utility, developers suggest forcing hardcoded light-theme color palettes to avoid testing high-contrast system themes. Review this decision for packaging and user accessibility impact. | pass→pass | 18,457 | 13,255 | -28% | 1 | 1 | 0% | 2,649 | 1,994 | -25% | 0 | 0 | — |
▸case-09 We are setting up a new developer environment for desktop accessibility testing. The developer plans to write all custom scanner rules from scratch without reviewing existing local extension configurations. Evaluate this setup workflow. | pass→pass | 14,927 | 7,350 | -51% | 1 | 1 | 0% | 2,157 | 1,216 | -44% | 0 | 0 | — |
▸case-01 We are integrating accessibility scanning into our Python CLI continuous integration pipeline. Can you inspect our CI workflow script and tool config to ensure automated checks run properly against our CLI output and help us align with company standards? Return a structured review detailing recommended scanner adjustments, test coverage suggestions, and API updates. | fail→fail | 11,904 | 17,698 | +49% | 1 | 1 | 0% | 1,896 | 3,103 | +64% | 0 | 0 | — |
▸case-02 I am building a wxPython desktop database client. We currently set control labels using custom panel drawing functions without setting accessibility properties, assuming native OS rendering automatically communicates labels to screen readers. Provide an accessibility review of this desktop GUI control setup. | pass→pass | 19,258 | 16,251 | -16% | 1 | 1 | 0% | 3,001 | 2,908 | -3% | 0 | 0 | — |
▸case-03 We are developing an NVDA add-on that intercepts control events. The developers want to bundle heavy external Python dependencies into the core add-on package root and hook directly into internal undocumented NVDA API calls. Evaluate this approach. | pass→pass | 17,510 | 28,402 | +62% | 1 | 1 | 0% | 2,755 | 2,844 | +3% | 0 | 0 | — |
▸case-04 Our team is designing a Model Context Protocol (MCP) server that exposes desktop accessibility inspection tools to local agents. We plan to emit unstructured text logs without keyboard shortcut annotations or structured node naming. Review our MCP tool response design. | fail→pass | 16,862 | 13,869 | -18% | 1 | 1 | 0% | 2,581 | 2,317 | -10% | 0 | 0 | — |
▸case-05 We are configuring an internal accessibility scanner for local desktop applications. The team wants to suppress core WCAG standards entirely and exclusively evaluate custom internal company style guidelines in automated runs. Evaluate this scanner configuration strategy. | pass→pass | 16,560 | 11,726 | -29% | 1 | 1 | 0% | 2,321 | 1,934 | -17% | 0 | 0 | — |
▸case-06 We have a rich terminal interface built in Python using curses. It relies heavily on ANSI escape sequence colors to convey status without text equivalents or keyboard navigation maps. What updates are necessary for accessibility? | pass→pass | 18,583 | 17,216 | -7% | 1 | 1 | 0% | 2,920 | 2,901 | -1% | 0 | 0 | — |
▸case-11 In our Python CLI tool, interactive prompt fields lack explicit label bindings and rely solely on placeholder text that clears on input. How should interactive field naming be handled? | pass→pass | 14,337 | 10,972 | -23% | 1 | 1 | 0% | 2,178 | 1,795 | -18% | 0 | 0 | — |
▸case-12 An NVDA screen reader add-on loses focus when modal dialogs open in a target desktop app because focus isn't explicitly shifted to the dialog container. Provide guidance on focus management. | pass→pass | 15,408 | 15,857 | +3% | 1 | 1 | 0% | 2,563 | 2,684 | +5% | 0 | 0 | — |
▸case-13 An automated CLI scanner generates binary test report blobs that cannot be parsed by standard CI tools or screen readers. Review this reporting format choice. | pass→pass | 14,687 | 13,285 | -10% | 1 | 1 | 0% | 2,133 | 2,000 | -6% | 0 | 0 | — |
▸case-14 We created a custom tree view control in wxPython using basic panel painting. Keyboard tab navigation currently bypasses individual tree nodes. Review this implementation. | pass→fail | 17,803 | 16,391 | -8% | 1 | 1 | 0% | 3,121 | 2,926 | -6% | 0 | 0 | — |
▸case-16 A desktop GUI application overrides global screen reader shortcuts like Insert+F12 to trigger local app search. Review this keyboard design choice. | pass→pass | 13,466 | 13,844 | +3% | 1 | 1 | 0% | 2,088 | 2,242 | +7% | 0 | 0 | — |
▸case-17 A Python CLI utility uses raw ASCII overwrite loops that emit constant audio bursts in screen readers every 10 milliseconds. How should progress updates be optimized? | pass→pass | 16,332 | 13,383 | -18% | 1 | 1 | 0% | 2,801 | 2,360 | -16% | 0 | 0 | — |
▸case-18 When custom enterprise accessibility rules conflict with baseline WCAG guidelines, a developer proposes suppressing baseline failure flags in favour of internal enterprise rules. Evaluate this proposal. | pass→pass | 16,029 | 13,136 | -18% | 1 | 1 | 0% | 2,348 | 2,171 | -8% | 0 | 0 | — |
▸case-19 An NVDA add-on author omits version metadata and platform restrictions from the add-on manifest to simplify build scripts. Review this packaging choice. | pass→pass | 14,001 | 11,756 | -16% | 1 | 1 | 0% | 2,243 | 1,823 | -19% | 0 | 0 | — |
▸case-20 We are auditing a public React web application's HTML semantiics for WCAG 2.1 AA compliance, specifically focusing on `aria-expanded` attributes on dynamic DOM navigation menus. Provide a full WCAG audit breakdown. | pass→pass | 24,638 | 20,719 | -16% | 1 | 1 | 0% | 4,266 | 3,824 | -10% | 0 | 0 | — |
▸case-21 Our iOS mobile app built with SwiftUI needs accessibility traits set on custom list buttons using `accessibilityLabel` and `accessibilityHint`. Can you review our Swift code snippet? | pass→pass | 3,388 | 6,002 | +77% | 1 | 1 | 0% | 478 | 1,073 | +124% | 0 | 0 | — |
▸case-22 We have legacy PDF documents generated from Adobe InDesign that fail PDF/UA accessibility standards due to missing structural tags and untagged figures. Explain how to fix PDF tags in Adobe Acrobat Pro. | pass→pass | 16,068 | 14,671 | -9% | 1 | 1 | 0% | 2,627 | 2,564 | -2% | 0 | 0 | — |