--- name: jetbrains-testing description: "Run or troubleshoot IntelliJ tests with `bt.cmd` or `tests.cmd`." license: "NOASSERTION" metadata: decimalai-skill-id: "771c83bd-0474-4ca6-92e9-77ca382946ea" decimalai-version-id: "efd82623-218f-484a-955b-254f42a47bb4" decimalai-version: "8" decimalai-body-sha256: "f88c1a07206b30bebb0621e48f3a81db99dc3773a54e588dec97dafba4d00dea" decimalai-name: "jetbrains/testing" decimalai-format: "1" decimalai-invocation: "model" decimalai-category: "testing" decimalai-source: "https://app.decimal.ai/s/jetbrains-testing@8/SKILL.md" --- # Testing Guide for IntelliJ IDEA ## Quick Start Run the tests of a module with bt first: ```bash ./community/tools/bt.cmd --module --filter ``` - `--module`: the JPS module that holds the test classes. Always use the module of the test itself. The module name is the name of the `.iml` file in the test directory, without the extension. - `--filter`: an FQN, `FQN#method`, or an all-lowercase package. A package value runs all the test classes in that package. In a community checkout, the wrapper is `./tools/bt.cmd`. ```bash # Single test class (FQN) ./community/tools/bt.cmd --module intellij.platform.util.tests \ --filter com.intellij.openapi.util.io.FileUtilLightTest # Specific test method ./community/tools/bt.cmd --module intellij.platform.util.tests \ --filter 'com.intellij.openapi.util.io.FileUtilLightTest#isAncestor' # All test classes of a package ./community/tools/bt.cmd --module intellij.platform.util.tests \ --filter com.intellij.openapi.util.io ``` bt runs a module when the migrated list names it, or when a `bt.json` area owns its directory. The [bt README](../../../tools/bt/README.md#modules) gives the resolution rules. ### bt options | Option | Effect | |--------|--------| | `--filter ` | Runs one class, one method, or one package of the target. | | `--no-cache` | Runs the tests again when Bazel has a cached result. | | `--dry-run` | Prints the resolved label, the filter, and the Bazel command. Runs nothing. | | `--json` | Writes one JSON object to stdout and nothing else. | | `--host ` | Runs the committed HEAD on another machine of the user through `bd-host` of Branch Deck. The `bd-host` skill (`bd-host skill install`) gives the rules. | | `-- --flaky_test_attempts=3` | Runs a failed test up to 3 times. `tests.cmd` uses `-Dintellij.build.test.attempt.count=3` for this. | | `-- --test_arg=--jvm_flag=-Dkey=value` | Sets a system property in the test JVM. `tests.cmd` uses `-Dpass.key=value` for this. | | `-- --test_arg=--jvm_flag=-Xmx8g` | Sets the heap of the test JVM. `tests.cmd` uses `-Dintellij.build.test.jvm.memory.options=-Xmx8g` for this. | bt gives all arguments after `--` to Bazel without a change. Run `./community/tools/bt.cmd --help` for all selectors and options. To attach a debugger, use `tests.cmd`. The section [Additional JVM options](#additional-jvm-options) shows the flags. | Exit code | Meaning | |-----------|---------| | 0 | All tests passed. | | 2 | Usage error, or bt refuses the module. | | 3 | Tests failed. | | 4 | Zero tests ran. | | 5 | The build failed before the tests. | | 6 | Infrastructure failure. | ### The check record and `bt build` A green bt run in a linked work tree, such as a Branch Deck lease, records the tree that it saw: one line in `bd-checks.jsonl` of the git directory. `bd lease pick` reads the record. A repository can set `pick_requires_check` in `.branchdeck.toml`, and then the pick refuses a lease whose tree has no green build or test. A run during which the files changed records nothing. A red run records nothing. Check the last tree with bt, not with a raw `./bazel.cmd build`. `bazel.cmd` gives the process to bazelisk, so it leaves no record. Commit or delete scratch files before the last check: a record of local changes counts only for a commit of the same files, and bt names those files on the stderr. ```bash ./community/tools/bt.cmd build //plugins/air/... # ./bazel.cmd build -k, one digest line, and the record ./community/tools/bt.cmd build //plugins/air/... --host studio ``` `bt build` exits 0 for a green build, 5 for a build failure, and 6 for an infrastructure failure. A `--host` run records the tree of the commit that ran on the host. The [bt README](../../../tools/bt/README.md#the-check-record) gives the record format. ### When bt refuses the module bt refuses a module whose tests do not run under Bazel yet. It exits with code 2 and prints the `tests.cmd` command. Run that command: ```bash ./tests.cmd --module --test ``` - `--module` — the JPS module containing the test classes (always use the test's own module). To find the module name, look at the `.iml` file in the test's directory — the module name is the `.iml` filename without the extension. - `--test` — FQN, wildcard pattern, or FQN#methodName ```bash # Single test class (FQN) ./tests.cmd --module intellij.cidr.compiler.custom.tests \ --test com.intellij.cidr.compiler.custom.CidrCustomCompilerReadTest # Wildcard pattern ./tests.cmd --module intellij.goland.tests \ --test com.goide.comments.*Test # Specific test method (wildcards cannot be used with #) ./tests.cmd --module intellij.cidr.compiler.custom.tests \ --test com.intellij.cidr.compiler.custom.CidrCustomCompilerReadTest#testSingleDefine # Multiple (semicolon-separated) ./tests.cmd --module intellij.platform.build.tests \ --test org.jetbrains.intellij.build.TestSelectorsTest#class selector;org.jetbrains.intellij.build.FileSetTest ``` **Simple class names like `MyTest` do NOT work** — always use FQN or wildcard (`*MyTest`). ### Why simple names fail Patterns are transformed (`*` → `.*`, `.` → `\.`) and matched against the fully qualified class name using `Pattern.matches()` (full-string match): | Pattern | Regex | Matches `org.example.MyTest`? | |---------|-------|-------------------------------| | `MyTest` | `MyTest` | NO — doesn't cover the package prefix | | `*MyTest` | `.*MyTest` | YES | | `org.example.MyTest` | `org\.example\.MyTest` | YES | | `org.example.*` | `org\.example\..*` | YES | ### Community tests Use `community/tests.cmd` for tests that belong to community-only modules: ```bash ./community/tests.cmd --module --test ``` ### API checks via Bazel In an Ultimate checkout, run `ApiCheckTest` directly through its Bazel test target. For a focused check, pass one or more comma-separated module names: ```bash ./bazel.cmd test //tests/ideaProjectStructure:projectStructureTests_test \ --test_filter=com.intellij.ideaProjectStructure.api.ApiCheckTest \ --test_arg=--jvm_flag=-Dapi.dump.test.modules.to.check=[,...] \ --test_output=summary \ --test_summary=detailed ``` Omit `--test_arg` to check all modules. Do not use `bazel run` for this target: it streams noisy test output and does not provide Bazel's test summary. This bt command is the same run with a digest: ```bash ./community/tools/bt.cmd //tests/ideaProjectStructure:projectStructureTests_test \ --filter com.intellij.ideaProjectStructure.api.ApiCheckTest \ -- --test_arg=--jvm_flag=-Dapi.dump.test.modules.to.check=[,...] ``` On failure, keep agent context small by extracting the failing dynamic-test names and messages from Bazel's JUnit report instead of printing the full `test.log`: ```bash xmllint --xpath '//testcase[failure or error]/@name | //testcase[failure or error]/*[self::failure or self::error]/@message' \ out/bazel-testlogs/tests/ideaProjectStructure/projectStructureTests_test/test.xml ``` ### Product layout and packaging changes When changing `ProductProperties`, `productImplementationModules`, product content descriptors, plugin/module-set packaging, or generated product layout XML, also run: ```bash ./community/tools/bt.cmd //build:all-products-packaging_test ``` The target pins the one class `AllProductsPackagingTest` and is tagged `manual`, so no `--filter` is needed, and no lane or wildcard pattern runs it. bt prints the digest, and Bazel writes `out/bazel-testlogs/build/all-products-packaging_test/test.xml`. The raw form is `./bazel.cmd test //build:all-products-packaging_test`. ### Windows PowerShell note When running `tests.cmd` from PowerShell, pass JVM `-D...` arguments via stop-parsing mode to avoid argument mangling: ```powershell ./tests.cmd --% -Dintellij.build.test.patterns=com.example.MyTest ``` Without `--%`, PowerShell can alter `-D...` arguments before they reach `tests.cmd`, which may lead to errors like `Could not find or load main class ...`. ## Separate Bazel Modules Some parts of the repository are standalone Bazel modules and must not use `tests.cmd` or `community/tests.cmd`. The tests of a module that matches a pattern in the migrated list run only under Bazel. The migrated list is `community/build/bazel-migrated-test-modules.txt`, or `build/bazel-migrated-test-modules.txt` in a community checkout. `tests.cmd` refuses such a module with an error that names the pattern. Run the module with bt: ```bash ./community/tools/bt.cmd --module --filter ``` When you have the Bazel label of the target, `./community/tools/bt.cmd