▸case-24 Write a Python repair hook that re-indexes missing Elasticsearch documents from Postgres. Ensure the hook can run repeatedly without triggering cascades of repair alerts or re-indexing already up-to-date documents. | pass→pass | 18,074 | 14,235 | -21% | 1 | 1 | 0% | 3,547 | 3,237 | -9% | 0 | 0 | — |
▸case-01 I'm building a background sync worker in Python that pushes updated experiment metadata to Braintrust whenever our primary event queue misses a write. Could you write the fallback update function so it safely updates Braintrust without accidentally wiping existing metadata if executed multiple times? | fail→pass | 17,276 | 6,352 | -63% | 1 | 1 | 0% | 3,212 | 1,322 | -59% | 0 | 0 | — |
▸case-02 We are setting up an automated reconciliation job to fix data drift between our SQL database and search index. Could you write the repair loop logic so it fills in missing index records without triggering secondary repair alerts or causing unnecessary churn? | fail→pass | 36,192 | 9,463 | -74% | 1 | 1 | 0% | 1,273 | 1,966 | +54% | 0 | 0 | — |
▸case-03 Write a primary checkout transaction in Python using PostgreSQL and SQLAlchemy that deducts inventory and creates an order record. Focus on standard primary write transactional safety and rollback on error. | pass→pass | 18,281 | 12,693 | -31% | 1 | 1 | 0% | 3,896 | 3,034 | -22% | 0 | 0 | — |
▸case-04 Write a read-only Prometheus metrics exporter endpoint in Python that calculates current active user count from Redis. Focus on fast read-side querying without mutating any state. | pass→pass | 15,759 | 9,847 | -38% | 1 | 1 | 0% | 3,092 | 2,053 | -34% | 0 | 0 | — |
▸case-05 Write an Alembic migration script in Python to add an index on user_id in the logs table. Focus on standard DDL migration steps. | pass→pass | 5,593 | 6,817 | +22% | 1 | 1 | 0% | 1,199 | 1,570 | +31% | 0 | 0 | — |
▸case-06 Write a Python fallback worker function that updates a Redis cache key with user settings whenever a cache miss occurs. Model writers often unconditionally call SET on every cache miss. How should the fallback write be written to prevent overwriting newer cached state or causing unnecessary cache writes? | pass→pass | 16,438 | 8,044 | -51% | 1 | 1 | 0% | 2,963 | 1,582 | -47% | 0 | 0 | — |
▸case-07 We have a nightly Python backup script that outputs report JSON files to a shared network mount as a fallback mechanism. To prevent corrupting partially written files if the script is interrupted or rerun concurrently, show how to write the file generation logic safely. | pass→pass | 15,852 | 10,371 | -35% | 1 | 1 | 0% | 2,898 | 2,166 | -25% | 0 | 0 | — |
▸case-08 I'm writing a Python helper function that syncs experiment metadata dictionary `{'status': 'completed', 'score': 0.95}` to Braintrust when an evaluation job finishes. What key option should be passed in the update payload to prevent overwriting existing keys in the dataset record? | pass→pass | 12,830 | 2,269 | -82% | 1 | 1 | 0% | 2,519 | 598 | -76% | 0 | 0 | — |
▸case-09 We have a secondary cleanup script that reconciles user subscription statuses in Postgres when the webhook handler fails. Write a Python snippet for the reconciliation query so it doesn't repeatedly trigger subscription webhooks or write when status is already up to date. | pass→fail | 11,806 | 6,513 | -45% | 1 | 1 | 0% | 2,143 | 1,488 | -31% | 0 | 0 | — |
▸case-10 Write a Node.js script that updates an application's local `config.json` file during container startup as a redundant fallback if environment variable injection failed. Ensure re-running the container multiple times won't corrupt the file or produce partial writes. | pass→pass | 19,736 | 9,698 | -51% | 1 | 1 | 0% | 3,205 | 2,071 | -35% | 0 | 0 | — |
▸case-11 Write a Python function that sends fallback telemetry metadata to a Braintrust logging span when primary logging times out. Make sure the metadata update does not wipe out previously attached span metadata. | fail→pass | 13,466 | 6,587 | -51% | 1 | 1 | 0% | 2,669 | 1,460 | -45% | 0 | 0 | — |
▸case-12 Write a Python API middleware that checks if a user's default settings row exists in PostgreSQL. If missing, it creates the default settings row. Ensure that high concurrency or repeated requests do not cause duplicate key errors or raise secondary repair alarms. | pass→pass | 16,968 | 9,601 | -43% | 1 | 1 | 0% | 3,225 | 2,038 | -37% | 0 | 0 | — |
▸case-13 Multiple worker threads in Python write fallback error reports to a single JSON state file on disk when central logging fails. Show how to implement the file write mechanism to prevent multiple threads from corrupting each other's writes. | pass→pass | 14,931 | 12,135 | -19% | 1 | 1 | 0% | 2,895 | 2,656 | -8% | 0 | 0 | — |
▸case-14 I am implementing a feedback handler in Python that updates Braintrust record scores when human reviewers submit scores asynchronously. How should the update dictionary be formatted to preserve existing record fields? | fail→pass | 13,838 | 4,123 | -70% | 1 | 1 | 0% | 2,165 | 854 | -61% | 0 | 0 | — |
▸case-25 When pushing fallback metadata to Braintrust in Python, what dictionary key-value pair must be included to perform a merge update instead of a full document overwrite? | fail→pass | 4,770 | 1,826 | -62% | 1 | 1 | 0% | 898 | 429 | -52% | 0 | 0 | — |
▸case-15 Write a Python worker that generates missing image thumbnails on S3 as a fallback when the image service misses an upload event. How should the worker check and write thumbnails to avoid redundant re-processing or overwriting existing thumbnails if run repeatedly? | pass→pass | 16,641 | 14,760 | -11% | 1 | 1 | 0% | 3,152 | 2,975 | -6% | 0 | 0 | — |
▸case-16 We are writing a Python script to reconcile user permissions between LDAP and our local database. The script runs every hour. If permissions are missing in local DB, it adds them. How should we write the loop to ensure running it 100 times in a row produces no unnecessary DB churn and no repeated alert triggers? | pass→pass | 15,720 | 9,114 | -42% | 1 | 1 | 0% | 2,822 | 1,625 | -42% | 0 | 0 | — |
▸case-17 Write a Python function that writes a local JSON state checkpoint `checkpoint.json` for a streaming job as a belt-and-suspenders persistence layer. How should file operations be structured so an abrupt system crash during the write leaves the previous valid checkpoint intact? | pass→pass | 15,619 | 7,861 | -50% | 1 | 1 | 0% | 2,985 | 1,730 | -42% | 0 | 0 | — |
▸case-18 Write a Python code snippet that updates a Braintrust experiment object with a `version: 'v2'` tag as a fallback sync step. How do you ensure this tag addition does not clear other existing tags on the experiment? | fail→pass | 10,273 | 5,069 | -51% | 1 | 1 | 0% | 1,783 | 1,134 | -36% | 0 | 0 | — |
▸case-19 Write a Python worker that processes dead-letter queue messages by re-inserting missing records into DynamoDB. What pattern should be used on the DynamoDB put operation to avoid overwriting items updated by other services? | pass→pass | 14,398 | 6,440 | -55% | 1 | 1 | 0% | 2,880 | 1,479 | -49% | 0 | 0 | — |
▸case-20 Write a Python background job that checks customer accounts with pending license status and activates them in PostgreSQL. How should the SQL update be structured so running it repeatedly does not touch already active rows or emit redundant activation emails? | pass→pass | 14,554 | 10,328 | -29% | 1 | 1 | 0% | 2,701 | 2,077 | -23% | 0 | 0 | — |
▸case-21 In our Python evaluator fallback pipeline, we need to push run metadata `{ 'retry_count': 1 }` to Braintrust. What parameter prevents wiping existing run metadata? | fail→pass | 20,640 | 2,825 | -86% | 1 | 1 | 0% | 3,916 | 644 | -84% | 0 | 0 | — |
▸case-22 Write a Python function that populates missing user session metadata in Memcached when primary session lookup fails. How should the function handle the fallback write to ensure it only writes if missing? | pass→pass | 14,073 | 7,896 | -44% | 1 | 1 | 0% | 2,816 | 1,539 | -45% | 0 | 0 | — |
▸case-23 Write a Python function that writes a fallback SQLite database snapshot file `backup.db` to disk. How do you ensure concurrent reads or interrupted writes never encounter a partially written file? | pass→pass | 15,251 | 8,569 | -44% | 1 | 1 | 0% | 2,827 | 1,771 | -37% | 0 | 0 | — |