▸case-01 You are processing a REST API JSON response containing transaction details: {'transaction_id': 'tx_998', 'amount': 149.50, 'merchant': 'Acme Supply', 'timestamp': '2025-02-01T10:30:00Z'}. Write a Python snippet to extract the merchant and amount fields directly into a dictionary using standard Python dictionary access. | fail→fail | 3,728 | 4,098 | +10% | 1 | 1 | 0% | 675 | 695 | +3% | 0 | 0 | — |
▸case-02 A user requests creating a marketing banner image showing an electric lawnmower on a green lawn. Choose between using an AI text-to-image generation model pipeline versus a vision-based visual document extraction model pipeline. | fail→fail | 7,368 | 5,963 | -19% | 1 | 1 | 0% | 1,232 | 959 | -22% | 0 | 0 | — |
▸case-03 You need to convert a 5-minute MP3 audio recording of a customer service call into a written text transcript. Choose the appropriate model architecture modality for this task. | fail→fail | 6,165 | 7,690 | +25% | 1 | 1 | 0% | 904 | 1,160 | +28% | 0 | 0 | — |
▸case-04 When sending a high-resolution scanned paper receipt image containing small 6pt printed text to OpenAI's Chat Completions API (/v1/chat/completions) with a vision model like gpt-4o, you want to ensure the model captures fine text details without losing resolution. A developer suggests omitting the detail parameter or setting it to low to save tokens. What value should be passed for the detail property inside the image content block? | fail→fail | 4,979 | 4,468 | -10% | 1 | 1 | 0% | 883 | 872 | -1% | 0 | 0 | — |
▸case-05 Before extracting structured form data using a vision-language model, digital multi-page PDF documents must be converted into image frames. Default PDF renderers often convert pages at standard screen display resolution of 72 DPI, which blurs fine text in scanned tables. What target resolution in DPI should be specified when converting PDF pages to images for optimal OCR and visual extraction quality? | fail→fail | 12,026 | 5,833 | -51% | 1 | 1 | 0% | 1,760 | 1,045 | -41% | 0 | 0 | — |
▸case-06 You are extracting structured receipt data (merchant name, total tax, and itemized array) from an image using a multimodal LLM. A team member proposes asking the model for unconstrained raw Markdown text and writing complex regular expressions to parse the result. How should you structure the API call to guarantee validated, type-safe JSON schema output from the model? | fail→fail | 12,257 | 12,734 | +4% | 1 | 1 | 0% | 2,169 | 2,583 | +19% | 0 | 0 | — |
▸case-07 When performing visual grounding or object detection on a 1920x1080 pixel screenshot using vision-language models like Qwen2-VL or Florence-2, the raw pixel coordinates need to be normalized. A developer proposes returning unscaled pixel dimensions [1920, 1080]. To what integer coordinate scale range are bounding box coordinates normalized in Qwen2-VL vision extraction outputs? | fail→fail | 7,782 | 5,746 | -26% | 1 | 1 | 0% | 1,366 | 1,087 | -20% | 0 | 0 | — |
▸case-08 You need to extract data from historical medical intake forms that feature handwritten doctor notes and pencil-checked square checkboxes. An engineer recommends using Tesseract 4.0 OCR. Which model approach handles zero-shot visual understanding of handwritten text and checkbox states effectively? | fail→fail | 16,811 | 11,297 | -33% | 1 | 1 | 0% | 2,306 | 1,844 | -20% | 0 | 0 | — |
▸case-09 You have a 10-page legal contract PDF that requires end-to-end visual analysis using a multimodal vision model. An engineer suggests merging all 10 page images into a single vertically concatenated panoramic image of 10,000 pixels height. How should the 10 page images be presented in the API request payload? | fail→fail | 11,079 | 10,368 | -6% | 1 | 1 | 0% | 2,141 | 1,835 | -14% | 0 | 0 | — |
▸case-10 You need to extract the underlying numerical data points from a bar chart image depicting quarterly revenue figures. A developer suggests asking the vision model for a single narrative paragraph describing the visual trends. How should the output prompt demand the chart data to facilitate immediate database insertion? | fail→fail | 9,731 | 9,765 | +0% | 1 | 1 | 0% | 1,594 | 1,702 | +7% | 0 | 0 | — |
▸case-11 Mobile phone photographs of receipts are uploaded to an automated visual extraction pipeline, but some images taken upside-down or sideways fail extraction because the vision model receives unrotated pixel matrices. A developer suggests trusting the vision model to infer rotation automatically. What preprocessing step must be applied to the image based on header metadata prior to model inference? | fail→fail | 6,020 | 3,943 | -35% | 1 | 1 | 0% | 1,035 | 685 | -34% | 0 | 0 | — |
▸case-12 When extracting a complex multi-column table with merged header cells from an architectural blueprint scan using a vision LLM, raw plain text output strips cell alignment. Which visual structure format should be requested in the model's response to preserve tabular hierarchy? | fail→fail | 11,158 | 7,868 | -29% | 1 | 1 | 0% | 1,589 | 1,295 | -19% | 0 | 0 | — |
▸case-13 You are constructing an HTTP request to an API for vision-based text extraction from an in-memory PNG image buffer. An engineer suggests sending a local file system file path string like file:///tmp/receipt.png inside the JSON image URL field. How must binary image data be encoded when passing it inline inside a data URI scheme? | fail→pass | 4,667 | 4,181 | -10% | 1 | 1 | 0% | 883 | 773 | -12% | 0 | 0 | — |
▸case-14 You are training a vision extraction model for layout analysis on document scans. Standard object detection formats format bounding boxes as [ymin, xmin, ymax, xmax]. A developer proposes reversing this to [xmin, ymax, xmax, ymin]. What is the standard coordinate ordering convention for normalized visual bounding boxes in vision models such as Florence-2 or LayoutLM? | fail→fail | 11,312 | 13,890 | +23% | 1 | 1 | 0% | 2,276 | 2,811 | +24% | 0 | 0 | — |
▸case-15 An application needs to extract only the signature block located in the bottom right corner of a 4K resolution document scan. A developer proposes sending the entire 4K image to the vision API on every request. How should the image be processed before API transmission to save tokens and eliminate noise? | fail→fail | 11,096 | 9,237 | -17% | 1 | 1 | 0% | 2,342 | 1,606 | -31% | 0 | 0 | — |
▸case-16 You are building an enterprise document extraction pipeline that ingests thousands of PDF invoices daily. An engineer suggests always rendering every PDF page to a bitmap image and running full vision LLM inference. What initial check should be performed on incoming PDF files before triggering image rendering and VLM extraction? | fail→fail | 11,656 | 8,935 | -23% | 1 | 1 | 0% | 1,911 | 1,503 | -21% | 0 | 0 | — |
▸case-17 A healthcare vision pipeline extracts patient identification numbers from photo IDs and sends the photos to a third-party vision API. An engineer suggests extracting the PII first and deleting the PII from the JSON response afterwards. How should sensitive visual PII regions be handled before transmitting images to external API endpoints? | fail→fail | 10,892 | 12,945 | +19% | 1 | 1 | 0% | 1,828 | 2,214 | +21% | 0 | 0 | — |
▸case-18 You are calculating token budgets for an OpenAI vision API request containing a 1024x1024 pixel image in high-detail mode. A developer suggests calculating tokens by taking the raw JPEG file size in bytes and dividing by 4. How does the API calculate image tokens for a high-detail 1024x1024 image? | fail→fail | 9,116 | 10,001 | +10% | 1 | 1 | 0% | 1,930 | 2,007 | +4% | 0 | 0 | — |
▸case-19 You are building a vision agent to visually identify click targets on a web page screenshot. A developer suggests relying solely on HTML element ID strings extracted from the DOM without visual coordinates. How should the vision extraction output specify interactive visual element locations for mouse click actions? | fail→fail | 11,351 | 14,703 | +30% | 1 | 1 | 0% | 1,893 | 2,008 | +6% | 0 | 0 | — |
▸case-20 You are designing a visual prompt to extract financial data from store receipts. A developer suggests a generic prompt: 'Tell me what is written on this paper'. How should the prompt be designed to ensure accurate visual key-value extraction? | fail→fail | 11,785 | 12,020 | +2% | 1 | 1 | 0% | 2,158 | 2,148 | -0% | 0 | 0 | — |
▸case-21 A high-volume processing system ingests 500,000 clear, perfectly oriented black-and-white printed text documents daily. A developer proposes routing every single page to high-cost multimodal Vision LLMs. What cost-effective alternative pipeline should be evaluated for clean printed text? | fail→fail | 13,635 | 12,066 | -12% | 1 | 1 | 0% | 2,474 | 1,941 | -22% | 0 | 0 | — |
▸case-22 Thermal store receipts often feature uneven lighting, background shadows, and paper crinkles. A developer proposes applying global fixed Otsu binarization, which turns faint thermal ink completely white. What preprocessing image operation or handling approach should be used instead for thermal paper text extraction? | fail→fail | 11,178 | 11,039 | -1% | 1 | 1 | 0% | 1,957 | 1,898 | -3% | 0 | 0 | — |