--- name: v8-v8-regression-testing description: "Use this skill when designing, writing, or minimizing regression tests (e.g., mjsunit) for V8 bug fixes" license: "NOASSERTION" metadata: decimalai-skill-id: "b624d0e2-2fc7-4024-b8e0-96184e4b1fdc" decimalai-version-id: "e7cb157d-4e34-430c-84d1-c588711836e4" decimalai-version: "3" decimalai-body-sha256: "4b9d3c2d8ed51f64c8634084ccc4accfae2b493d2a219dcc51d74160bf52ab78" decimalai-name: "v8/v8-regression-testing" decimalai-format: "1" decimalai-invocation: "model" decimalai-category: "testing" decimalai-source: "https://app.decimal.ai/s/v8-v8-regression-testing@3/SKILL.md" --- # Skill: V8 Regression Testing Use this skill to design, write, minimize, and refine high-quality regression tests in the V8 codebase (typically under `test/mjsunit/` or its component-specific subfolders like `compiler/`, `turboshaft/`, `maglev/`, etc.). ## Core Mandates - **Mandatory Reproducer Rule**: Unless explicitly specified otherwise by the user, **every single bug fix in V8 MUST be uploaded with a working reproducer (regression test)**. Uploading just the fix alone is NEVER acceptable. - **Verify Every Test Case Fails Without the Fix**: Always check that every regression test (and every individual test case within it) a) actually exercises the specific code path/fix being tested and b) fails without the fix and passes with the fix. - **Semantic Quality**: V8 regression tests must be **semantic (root-cause-driven)** rather than just **syntactic (fuzzer-driven)**. A high-quality test case is a clear, human-readable, and robust demonstration of the specific logical bug or compiler invariant under test, rather than a fragile, black-box crash script. - **CRITICAL REPRODUCER CONSTRAINT**: **NEVER** provide a reproducer that does not actually trigger the issue to the user without **explicitly** telling them about it. ______________________________________________________________________ ## The "From-Scratch" Reproducer Philosophy When constructing a reproducer for a bug, **always prefer writing it from scratch based on a deep conceptual understanding of the issue, rather than starting from a fuzzer-generated crash script and trying to minimize it.** ### Why the Post-Hoc Fuzzer Minimization Trap Fails: 1. **The Black Box Dilemma**: Starting with a fuzzer-generated block of code means you are constantly guessing which strange construct is necessary. You are highly likely to retain useless nesting (`__wrapTC`), redundant try-catches, and magic numbers because you are afraid to break the fragile reproduction state. 2. **Fragility**: Fuzzer scripts often rely on incidental timings, engine heuristics, or side effects to trigger a crash. If optimization heuristics change slightly in a future V8 release, the fuzzer reproducer will silently break and stop testing the intended bug path. ### The "From-Scratch" Synthesis Approach: 1. **Identify the Root Cause**: Analyze the compiler phase graph, the crash backtrace, and the failing `DCHECK` or invariant to understand the exact abstract state machine failure (e.g., "an incorrect type representation selection, a specific Map transition mismatch, or a loop-carried side-effect"). 2. **Begin with a Blank File**: Open an empty editor. Do not copy-paste the fuzzer script. 3. **Target the State Directly**: Write the absolute minimum JavaScript necessary to construct that logical state: - Use clean, structured object definitions and allocations to establish target internal structures (such as distinct Maps or Shapes). - Control the IC feedback vector cleanly by warming it up with specific types. - Explicitly trigger the compiler pass with intrinsic calls (e.g., `%PrepareFunctionForOptimization`, `%OptimizeFunctionOnNextCall`). 4. **Result**: You produce a clean, robust, perfectly commented, 15-line reproducer that explicitly tests the underlying logic. It is self-documenting, easy to maintain, and impervious to unrelated engine optimization tweaks. ______________________________________________________________________ ## Debugging a Failing Reproducer (The Scientific Loop) When writing a reproducer from scratch, it is common for the first attempt to **not** trigger the crash. When this happens, do not give up or fall back to blindly copying the fuzzer script. Instead, treat the reproducer's failure as a debugging task and enter a scientific feedback loop. ### The Feedback Loop: 1. **Trace the Execution**: Use V8's extensive set of tracing flags to inspect the JIT/engine pipeline (eg, `--trace-turbo-graph`, `--trace-ic`, `--trace-deopt`...), and insert custom instruction (eg, using `PrintF`) as needed if the existing flags are too coarse. 2. **Detect the Detour**: Identify exactly where the execution path diverged from your expectation. For instance: - *Did the function deoptimize prematurely due to an unexpected type feedback change?* - *Did load elimination optimize away a read that was supposed to trigger a side effect?* - *Did a map transition occur too early?* 3. **Analyze the Cause**: Figure out *why* V8 chose that detour. - What invariant, heuristic, or state in your JavaScript code caused the compiler to take a different optimization path? 4. **Heal and Steering**: Modify your JavaScript reproducer to block the detour and steer the compiler back onto the target path (e.g., altering map setups, introducing fake side effects to prevent load-elimination, or adjusting argument warming patterns). [!IMPORTANT] **The Ultimate Conceptual Checkpoint**: If you repeatedly fail to construct a clean, working reproducer from scratch, **this almost always means your conceptual understanding of the bug is either incorrect or too shallow.** Do not waste days trying to brute-force random JavaScript tweaks. When this checkpoint is hit, you must **go back to the drawing board**, challenge what you think you know about the bug, and deepen your understanding. ______________________________________________________________________ ## Syntactic Principles of a Good Regression Test ### 1. Minimal & "Leaf" Compiler Flags - **Strict Flag Minimization**: Every flag included in `// Flags: ...` must be thoroughly understood and strictly necessary to reproduce the issue. Remove all useless or redundant flags. - **Prefer "Leaf" Flags over "Compound" Flags**: - **Bad**: Using `--jit-fuzzing`, `--fuzzing`, or `--experimental-fuzzing` because they imply a massive, ever-changing bundle of underlying behaviors. - **Good**: Identifying and specifying the exact leaf flags responsible (e.g., `--no-lazy-feedback-allocation`, `--homomorphic-ic`, `--stress-concurrent-inlining-attach-code`). ### 2. Human-Friendly Naming Avoid default fuzzer variable and function names (`__f_0`, `__v_10`, `v17`, `a6`, `a7`). - Use **descriptive names** for semantic concepts (`large_arr`, `global_var`, `testGenerator`). - Use **clean, standard placeholders** for simple generic logic (`foo`, `bar`, `obj`, `f`, `x`, `y`). - Maintain cognitive load at an absolute minimum. ### 3. Absolute Minimization of Code & Boilerplate - Remove every parameter, variable, block, or helper that does not directly contribute to reproducing the bug. - Never leave unused parameters in function signatures, unless absolutely required to reproduce the issue. - Do not include catch-all blocks (`catch (e) {}`) or dummy wrapper functions (`__wrapTC`) unless they are semantically required to trigger the crash path. - **Do not explicitly load `mjsunit.js`**: `mjsunit` tests run via `tools/run-tests.py` automatically include `test/mjsunit/mjsunit.js`. Never include `d8.file.execute('test/mjsunit/mjsunit.js')` in the test file itself. (When executing the test manually via `d8`, pass `test/mjsunit/mjsunit.js` on the command line instead). ### 4. Deterministic Optimization Control - Avoid spinning CPU-burning loops (e.g., `for (let i = 0; i < 100000; i++)`) to trigger JIT tier-up, unless the bug specifically relates to loop-scaling or profiling feedback. - Use V8 intrinsic compiler controls to force deterministic optimization: - `%PrepareFunctionForOptimization(foo);` - `%OptimizeFunctionOnNextCall(foo);` - Make sure `--allow-natives-syntax` is included in the Flags header when using percent (`%`) intrinsics. - **`%OptimizeFunctionOnNextCall` and Maglev Variants**: Test runner variants run tests both with and without `--optimize-on-next-call-optimizes-to-maglev` (e.g., `default` vs. `stress_maglev` / `maglev_no_turbofan`). For generic regression tests using `%OptimizeFunctionOnNextCall`, even when testing a Maglev regression, omit `--optimize-on-next-call-optimizes-to-maglev` from `// Flags:` in the test file and verify that the test fails without the fix under `--variants=stress_maglev` (or by passing `--optimize-on-next-call-optimizes-to-maglev` on the command line). ### 5. Rely on Test Runner Variants for Execution Coverage - V8's test runner automatically executes test suites across different variants (such as baseline execution, optimizing tiers, and stress modes). - Avoid manually forcing execution across multiple tiers or duplicating invocations solely to cover different compiler pipelines, unless the test specifically exercises tier-up dynamics, on-stack replacement, or deoptimization transitions between them. ### 6. Component-Specific Subdirectories - Place new `mjsunit` regression tests in the most specific component subdirectory (e.g., `test/mjsunit/compiler/`, `test/mjsunit/maglev/`, `test/mjsunit/turbolev/`, `test/mjsunit/turboshaft/`, `test/mjsunit/wasm/`) rather than the generic catch-all `test/mjsunit/regress/` directory, and test features in the configuration where they are enabled by default. - When inheriting or rebasing a regression test from an earlier draft, audit it against all guidelines in this skill before uploading.