▸case-01 We are running a Python 3 unit test suite in CI where dictionary iteration order causes intermittent test failures across runner nodes. Developers suggest adding dict.keys() sorting inside every single application function. What environment variable should be configured in the test runner job to enforce predictable hash seed behavior across Python processes? | pass→pass | 4,940 | 4,980 | +1% | 1 | 1 | 0% | 798 | 779 | -2% | 0 | 0 | — |
▸case-02 Our PyTorch model training test suite produces slightly different loss values on the exact same CUDA GPU across separate runs, even though torch.manual_seed(42) is called. Engineers want to replace PyTorch with CPU-only NumPy calculations. What PyTorch function should be called to force underlying CUDA convolution algorithms to use deterministic implementations? | pass→pass | 6,645 | 5,902 | -11% | 1 | 1 | 0% | 1,227 | 1,173 | -4% | 0 | 0 | — |
▸case-03 A test passes when run individually but occasionally fails when running the whole suite. Developers want to manually run pytest by hitting up-arrow and Enter 50 times in their terminal to reproduce the race condition. What CLI flag from the pytest-repeat plugin should be used to execute a target test 100 times in a single automated command? | pass→pass | 2,931 | 2,337 | -20% | 1 | 1 | 0% | 574 | 435 | -24% | 0 | 0 | — |
▸case-04 A JavaScript integration test fails depending on what time of day or timezone the developer runs it because it calls new Date(). The team wants to sleep for 24 hours in the test script or change the system OS time before running npm test. What method should be called in Jest to freeze time deterministically without modifying the host OS time? | pass→pass | 5,555 | 4,885 | -12% | 1 | 1 | 0% | 1,037 | 1,014 | -2% | 0 | 0 | — |
▸case-05 Our CI pipeline runs reproducibility tests inside Docker containers using 'FROM python:3.11-slim' in the Dockerfile. Yesterday's build produced different package checksums than today's build despite no code changes. A junior dev suggests running apt-get update at the start of every test run. How should the Dockerfile base image reference be modified to guarantee exact bit-for-bit container environment reproducibility? | pass→pass | 10,411 | 9,940 | -5% | 1 | 1 | 0% | 1,787 | 1,831 | +2% | 0 | 0 | — |
▸case-06 When running automated tests on CI workers, the test runner executes npm install. Occasionally, a transitive dependency release causes tests to fail unexpectedly on CI while passing on developer laptops. Teammates suggest deleting package-lock.json and running npm update. What npm command should be executed in CI to enforce strict adherence to the lockfile without updating dependencies? | pass→pass | 4,222 | 4,317 | +2% | 1 | 1 | 0% | 779 | 868 | +11% | 0 | 0 | — |
▸case-07 An automated integration test suite relies on live HTTP calls to a third-party weather API. Tests fail intermittently due to network latency, rate limits, and live data changes. Developers suggest mocking every HTTP call manually with custom inline stub functions in every test file. What recording cassette pattern should be used to capture real HTTP responses once and replay them deterministically in test runs? | pass→pass | 12,698 | 9,218 | -27% | 1 | 1 | 0% | 2,252 | 1,624 | -28% | 0 | 0 | — |
▸case-08 A file batch processor test suite works on macOS but fails deterministically on Linux CI runners. The function uses os.listdir('/path/to/files') to process files in order. A developer suggests re-formatting the Linux disk to HFS+. How should the application code or test assertion be adjusted to handle non-deterministic filesystem directory traversal order across operating systems? | pass→pass | 10,125 | 8,204 | -19% | 1 | 1 | 0% | 1,875 | 1,566 | -16% | 0 | 0 | — |
▸case-09 A financial calculation test asserts result == 0.30000000000000004 after floating-point operations, but on ARM64 CI instances it returns 0.3. The developer wants to cast all floating point numbers to strings and perform string truncation to 2 characters before comparing. What assertion function in pytest should be used to test floating-point equality within defined tolerances? | pass→pass | 3,959 | 4,731 | +19% | 1 | 1 | 0% | 681 | 943 | +38% | 0 | 0 | — |
▸case-10 We are compiling C binaries and building Docker images as part of artifact reproducibility testing. Compiling the exact same source code 5 minutes apart produces differing SHA-256 hashes due to embedded compilation timestamps. Developers suggest stripping all headers from the binary post-compile. What standard environment variable should be exported to force compiler and packaging tools to use a fixed build timestamp? | pass→pass | 5,932 | 5,892 | -1% | 1 | 1 | 0% | 1,158 | 1,080 | -7% | 0 | 0 | — |
▸case-11 When running unit tests sequentially with pytest, all tests pass. When running in parallel with 'pytest -n auto', random seed state leaks across test threads running on the same worker, causing random failures. Developers want to disable parallel test execution entirely. How should random seeds be managed in pytest fixtures to ensure isolated PRNG state per test run? | pass→pass | 15,768 | 11,370 | -28% | 1 | 1 | 0% | 2,856 | 2,542 | -11% | 0 | 0 | — |
▸case-12 A test generates a JSON file and asserts its SHA256 hash against a golden reference. The test fails on Python 3.11 when comparing serialized strings because object key ordering in generated output varies across dictionary constructions. A developer wants to rewrite the test to compare unparsed string substrings. What parameter in json.dumps() ensures deterministic key ordering in output strings? | pass→pass | 2,660 | 2,613 | -2% | 1 | 1 | 0% | 511 | 475 | -7% | 0 | 0 | — |
▸case-13 Database integration tests leave behind dirty records in PostgreSQL, causing subsequent tests in the suite to fail due to duplicate key errors. A teammate proposes running TRUNCATE TABLE on all tables in a teardown script after every test run, which slows down the suite significantly. What transaction pattern should be used in test fixtures to ensure fast and deterministic clean state per test? | pass→pass | 10,919 | 12,709 | +16% | 1 | 1 | 0% | 1,830 | 2,097 | +15% | 0 | 0 | — |
▸case-14 A string sorting test passes on developer workstations set to en_US.UTF-8 but fails on minimal Linux CI container environments set to POSIX/C locale because string collation rules differ. Engineers propose writing custom ASCII character sorting loops in Python. What environment variable setting should be configured in the CI container environment to standardize locale collation behavior? | fail→pass | 6,582 | 7,794 | +18% | 1 | 1 | 0% | 1,273 | 1,310 | +3% | 0 | 0 | — |
▸case-15 A legacy data pipeline test uses legacy numpy.random.seed(123) globally, but multi-threaded data processing functions instantiate new PRNG instances, leading to non-deterministic data generation. Developers propose replacing NumPy with hardcoded CSV mock files. How should NumPy's modern random generation API (np.random.default_rng) be initialized to ensure thread-isolated deterministic random streams? | pass→pass | 11,837 | 13,868 | +17% | 1 | 1 | 0% | 2,310 | 2,004 | -13% | 0 | 0 | — |
▸case-16 An application snapshot test includes the current git commit hash in output headers, causing the snapshot to fail on every new commit. Developers want to disable git integration in production code when running tests. How should dynamic system metadata like git commit hashes or build versions be handled in snapshot test comparisons? | pass→pass | 13,201 | 12,747 | -3% | 1 | 1 | 0% | 2,206 | 2,234 | +1% | 0 | 0 | — |
▸case-17 Tests in a suite pass when run in alphabetical file order, but fail when run in arbitrary order because Test B implicitly depends on state mutated by Test A. Developers suggest forcing pytest to run alphabetically using custom file naming like 01_test.py, 02_test.py. What test runner plugin approach should be used to continuously randomize test execution order in CI to surface order-dependent state leaks? | pass→pass | 10,439 | 8,092 | -22% | 1 | 1 | 0% | 1,890 | 1,376 | -27% | 0 | 0 | — |
▸case-18 A machine learning inference test suite running on a 64-core CPU produces non-deterministic floating-point outputs across runs due to variable thread scheduling in OpenMP and BLAS matrix operations. Developers suggest running tests on single-core hardware instances. What environment variable should be set to restrict OpenMP operations to a deterministic single thread during testing? | pass→pass | 3,101 | 3,208 | +3% | 1 | 1 | 0% | 612 | 592 | -3% | 0 | 0 | — |
▸case-19 A Python unit test for a subscription expiration check calls datetime.now() and fails when run at midnight UTC. A junior dev suggests inserting time.sleep(1) inside a loop until the time matches expected values. What Python library pattern should be used to lock datetime.now() to a static reference timestamp throughout the test execution? | pass→pass | 11,803 | 8,565 | -27% | 1 | 1 | 0% | 2,146 | 1,657 | -23% | 0 | 0 | — |
▸case-20 End-to-end visual regression tests using Playwright capture flakiness because CSS animations and transitions complete at slightly different frame timings during screenshot capture. Developers propose adding a 5-second page.waitForTimeout(5000) before every screenshot. How should CSS animations be disabled in Playwright to achieve deterministic visual snapshots? | pass→pass | 11,639 | 10,565 | -9% | 1 | 1 | 0% | 2,181 | 2,143 | -2% | 0 | 0 | — |
▸case-21 We need to design a load test script using k6 to verify that our payments API service can sustain 500 requests per second under peak traffic load. What k6 options structure should be defined to configure virtual users and ramp-up stages for this throughput benchmark? | pass→pass | 12,241 | 10,012 | -18% | 1 | 1 | 0% | 2,486 | 2,059 | -17% | 0 | 0 | — |
▸case-22 We want to configure OWASP ZAP in our CI pipeline to perform an automated baseline security scan against our staging REST API endpoints for common vulnerabilities like SQL injection and cross-site scripting. How should the zap-baseline.py command be invoked? | pass→pass | 15,710 | 11,074 | -30% | 1 | 1 | 0% | 2,907 | 2,247 | -23% | 0 | 0 | — |
▸case-23 Write a Gherkin BDD feature file scenario for testing a user authentication login page, covering successful login with valid credentials and error handling for invalid passwords. | pass→pass | 8,148 | 6,030 | -26% | 1 | 1 | 0% | 1,637 | 1,133 | -31% | 0 | 0 | — |