▸case-01 I need a complete Python SDK created from my OpenAPI specification located at `./specs/payments_api.json`. Please generate the full source code package, the `pyproject.toml` file configured for Poetry, a test suite using `pytest`, Sphinx-ready documentation, and some runnable example scripts. The SDK should target Python 3.8+ and support both async and sync calls with strict type hints. | fail→fail | 38,796 | 49,276 | +27% | 1 | 1 | 0% | 8,271 | 8,697 | +5% | 0 | 0 | — |
▸case-02 We have a GraphQL service for weather telemetry and want a dedicated client library. Can you build the Python package for us based on `./schema/weather.graphql`? We need the client source code, dependency/build configuration files, pytest unit tests, API documentation, and sample usage scripts. Make sure it uses `httpx` for async requests and includes type annotations. | fail→pass | 39,338 | 38,546 | -2% | 1 | 1 | 0% | 8,263 | 8,689 | +5% | 0 | 0 | — |
▸case-03 Please build a production-ready Python SDK for our custom user management REST API using the schema in `./api/users_spec.yaml`. I want the generated SDK files, type stubs, tests, packaging configuration for setuptools, and docs with usage examples. It should support async context managers and run on Python 3.10+. | fail→fail | 34,017 | 31,506 | -7% | 1 | 1 | 0% | 8,258 | 8,411 | +2% | 0 | 0 | — |
▸case-04 We are building a payment handling server using FastAPI. Please implement the backend API endpoint function in `app/routers/payments.py` that receives HTTP POST requests at `/v1/charge`, verifies the JSON request body, and returns a JSON response. | pass→pass | 18,252 | 19,984 | +9% | 1 | 1 | 0% | 2,835 | 3,405 | +20% | 0 | 0 | — |
▸case-05 We already built our Python package distribution files in `dist/`. Write a GitHub Actions CI workflow job step that uses `twine upload` with a PyPI API token secret to publish the wheel files to PyPI. | pass→pass | 6,871 | 13,231 | +93% | 1 | 1 | 0% | 1,393 | 1,987 | +43% | 0 | 0 | — |
▸case-06 We need a low-level C extension module using PyO3 in Rust to perform fast binary parsing of telemetry packet buffers in Python. Implement the Rust `#[pyfunction]` entry point and C-extension module bindings. | pass→pass | 26,152 | 30,713 | +17% | 1 | 1 | 0% | 4,327 | 6,022 | +39% | 0 | 0 | — |
▸case-07 Write a high-level API wrapper method in Python for a user profile endpoint. A lot of tools default to reStructuredText `:param user_id:` markup. Implement the method with comprehensive type annotations and detailed docstrings detailing `user_id`, `include_deleted`, and returning `UserProfile`. | fail→pass | 14,299 | 13,501 | -6% | 1 | 1 | 0% | 1,906 | 3,158 | +66% | 0 | 0 | — |
▸case-08 Write a Python data model class representing an API filter payload with optional search strings and a list of tag strings. Model developers often use Python 3.10 pipe types (`str | None`) and native `list[str]` annotations without imports, but this needs to run on standard Python 3.8 runtime environments without syntax errors. | pass→fail | 17,583 | 16,032 | -9% | 1 | 1 | 0% | 2,381 | 3,436 | +44% | 0 | 0 | — |
▸case-09 Implement a client connection class `ApiClient` that manages an internal HTTP session. Developers often write a custom `.close()` method and rely on callers to remember calling it manually. Provide native context manager lifecycle support for both synchronous and asynchronous execution blocks. | pass→pass | 31,286 | 28,454 | -9% | 1 | 1 | 0% | 5,954 | 5,667 | -5% | 0 | 0 | — |
▸case-10 Create the API client class interface for an order processing service. Many python developers choose to write only async methods or force callers to use `asyncio.run()`. Generate the client interface so it accommodates both synchronous scripts and asynchronous event loops. | pass→pass | 21,478 | 25,540 | +19% | 1 | 1 | 0% | 4,297 | 5,037 | +17% | 0 | 0 | — |
▸case-11 Create the response parsing models for an inventory API payload. Standard Python code often relies on raw `dict` indexing or standard `dataclasses` without runtime validation. Build the data models so incoming JSON dictionaries are validated against schema types on instantiation. | pass→pass | 17,142 | 22,967 | +34% | 1 | 1 | 0% | 3,564 | 4,905 | +38% | 0 | 0 | — |
▸case-12 Create the build and distribution configuration file for an API client library named `acme-client`. Standard practice in legacy codebases is to write a dynamic `setup.py` script. Produce a modern declarative package configuration file conforming to PEP 517. | pass→pass | 17,151 | 12,442 | -27% | 1 | 1 | 0% | 2,539 | 3,013 | +19% | 0 | 0 | — |
▸case-13 We have an SDK module `dyn_api.py` that dynamically generates methods at runtime. Standard static analysis tools fail on these dynamic calls. Write the static stub file that IDEs and static checkers use to inspect method signatures without running code. | pass→pass | 13,018 | 28,312 | +117% | 1 | 1 | 0% | 2,443 | 5,178 | +112% | 0 | 0 | — |
▸case-14 Write the package configuration file for a Python library where heavy HTTP dependencies like `httpx` or CLI dependencies are optional addons rather than required core packages. Common setup files list all requirements under mandatory dependencies. Declare these optional feature sets correctly. | pass→pass | 19,153 | 19,061 | -0% | 1 | 1 | 0% | 2,760 | 3,279 | +19% | 0 | 0 | — |
▸case-15 Write an authentication token manager class for a Python client library. Untyped parameters or implicit `Any` return types are common in dynamic Python code. Provide full, explicit type annotations for every method parameter and return value to satisfy strict mypy analysis. | pass→pass | 21,227 | 32,144 | +51% | 1 | 1 | 0% | 4,369 | 6,277 | +44% | 0 | 0 | — |
▸case-16 Write a test suite file `tests/test_client.py` for testing an asynchronous client's `get_account` method. Standard `unittest` style requires boilerplate event loop creation inside tests. Write the tests using modern Python testing practices for async calls. | pass→pass | 11,573 | 21,064 | +82% | 1 | 1 | 0% | 2,353 | 3,682 | +56% | 0 | 0 | — |
▸case-17 Set up the documentation project configuration for a Python SDK so that documentation tools can auto-generate API references directly from Google-style docstrings in the source code. | pass→pass | 14,371 | 25,922 | +80% | 1 | 1 | 0% | 2,784 | 4,626 | +66% | 0 | 0 | — |
▸case-18 Write an HTTP retry handler for an SDK library that records retry attempts and status code warnings. Many developers print log messages directly to `sys.stdout` using `print()`. Implement standard Python logging mechanism. | pass→pass | 20,865 | 31,405 | +51% | 1 | 1 | 0% | 3,558 | 6,134 | +72% | 0 | 0 | — |
▸case-19 Design error handling for a payment SDK that encounters HTTP 401 unauthorized and HTTP 429 rate limit responses. Developers often throw generic `Exception` or standard `RuntimeError`. Design a custom exception structure so SDK consumers can catch all SDK-specific errors using a single base exception class. | pass→pass | 24,088 | 26,627 | +11% | 1 | 1 | 0% | 3,759 | 5,032 | +34% | 0 | 0 | — |
▸case-20 Write the `__init__.py` module for a library `acme_sdk` containing internal helper functions `_internal_parse()` alongside public classes `AcmeClient` and `AcmeException`. Prevent internal helper functions from being exported when consumers run `from acme_sdk import *`. | pass→pass | 11,018 | 12,631 | +15% | 1 | 1 | 0% | 1,079 | 1,805 | +67% | 0 | 0 | — |
▸case-21 Implement an HTTP adapter class for sending async API requests in Python. Standard legacy code uses `urllib.request` or `requests` wrapped in threads. Implement the request transport using modern asynchronous HTTP client libraries. | pass→pass | 21,560 | 28,652 | +33% | 1 | 1 | 0% | 3,407 | 5,661 | +66% | 0 | 0 | — |
▸case-22 Write a standalone usage example script `examples/demo.py` for an API SDK that shows how to initialize the client and make a request. Provide clear execution structure so the script can be run directly from the command line. | pass→pass | 17,669 | 22,128 | +25% | 1 | 1 | 0% | 2,342 | 4,096 | +75% | 0 | 0 | — |
▸case-23 Create a package configuration file for an SDK named `metrics-sdk` using Poetry, targeting Python 3.8+ and including dependencies on HTTP transport and validation libraries. Standard pip setuptools uses setup.cfg or setup.py. Construct the configuration file using Poetry's dependency structure. | pass→pass | 11,387 | 15,762 | +38% | 1 | 1 | 0% | 1,328 | 2,510 | +89% | 0 | 0 | — |