---
name: dry-software/nib
source: https://app.decimal.ai/s/dry-software-nib@1/SKILL.md
source_sha256: 737b6ee59b46
---

You are performing desktop automation using `nib` (nut.js Instrumentation Bundle).

Execute the following task: $ARGUMENTS

---

# How to use nib

Run commands via Bash: `nib <command> [options]`

All commands return JSON to stdout. Always parse and check the `ok` field:
```json
{"ok":true, "command":"...", "data":{...}}
{"ok":false,"command":"...", "error":{"code":"...","message":"...","suggestion":"..."}}
```

If a command fails, read the error message and suggestion carefully before retrying.

Before starting, run `nib status` and check `data.permissions`. On macOS, `accessibility` and `screenRecording` must both be `true` — otherwise a11y and window-title commands fail with `PERMISSION_DENIED`. If a permission is missing, run `nib request-permissions` once to trigger the system prompt, tell the user to approve it (and restart the host app), and wait for their confirmation. Do not retry a failed command or re-run `request-permissions` in a loop — granting requires user action, and macOS shows the prompt only once per app.

---

# Workflow

**Always prefer accessibility-based element interaction over coordinate-based mouse clicks.** Element refs are robust against layout changes and resolution differences.

## Standard interaction flow

```bash
# 1. Discover and focus the target window
nib list-windows
nib focus-window "App Name"

# 2. Take an accessibility snapshot (interactive-only is faster)
nib snapshot --window "App Name" -i

# 3. Read the refs from the snapshot output, then interact
nib click-element @btn:Save --window "App Name"
```

## After UI changes (navigation, dialog open/close)

```bash
# Use diff to see what changed (faster than a full re-snapshot)
nib diff --window "App Name"

# Or take a fresh snapshot if the UI changed significantly
nib snapshot --window "App Name" -i
```

---

# Element refs

Snapshots assign stable refs to every interactive element. Refs encode the element type and label:

| Ref | Meaning |
|-----|---------|
| `@btn:Save` | Button labeled "Save" |
| `@txt:Username` | Text field labeled "Username" |
| `@chk:Remember` | Checkbox labeled "Remember" |
| `@btn~2:Submit` | 2nd button labeled "Submit" (disambiguated) |
| `@btn~1` | 1st untitled button |

**Type abbreviations:** btn=button, chk=checkbox, cmb=combobox, lnk=link, mnu=menuitem, rad=radiobutton, sld=slider, tab=tab, txt=textfield, txa=textarea, tri=treeitem, cel=cell, edt=editfield, lst=listitem, tgl=togglebutton, swt=switch, pbtn=popupbutton, mbtn=menubutton, mbar=menubaritem, dsc=disclosuretriangle, dock=dockitem, inc=incrementor, sbtn=splitbutton, clr=colorwell, spn=spinner, dat=dataitem, cmnu=checkmenuitem, rmnu=radiomenuitem

---

# Commands

## Window targeting

Most commands that operate on a window accept multiple ways to identify the target:

| Option | Description |
|--------|-------------|
| `--window <title>` | Match the full window title exactly (case-sensitive) |
| `--window-contains <text>` | Match window title by case-insensitive substring |
| `--window-regex <pattern>` | Match window title by case-insensitive regex |
| `--app <name>` | Filter by application name |
| `--pid <id>` | Filter by process ID |
| `--path <path>` | Filter by executable path |
| `--bundle-id <id>` | Filter by bundle identifier (macOS) |
| `--active-window` | Target the currently active window |

Filters can be combined (`--active-window` excluded). A window selector is always required — there is no fallback to the active window, and a selector that matches nothing (or matches several windows) is an error. On multiple matches the error lists the candidates; narrow with an additional filter or `--pid`.

## Window management

```
nib list-windows [--full] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib active-window [--full]
nib focus-window [title] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib window-region [title] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib resize-window [title] <width> <height>
nib move-window [title] <x> <y>
nib minimize-window [title]
nib restore-window [title]
nib window-elements [title] [--max N]
```

- `--window` and positional `[title]` arguments match the full title exactly; use `--window-contains` / `--window-regex` for partial titles.
- `--full` on `list-windows` and `active-window` includes owner details (process ID, name, path, bundle ID) and window state.
- All window commands support window targeting options.

## Accessibility & element discovery

```
nib snapshot [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>] [-i] [--tree] [--compact] [--max N]
nib diff [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib find-element [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>] [--id <p>] [--type <t>] [--title <p>] [--role <r>] [--value <v>] [--class-name <p>] [--help-text <p>] [--selected-text <p>]
nib get-element <ref> --window <title> [--property title|value|role|type|region|states|all]
nib check-element <ref> --window <title> [--property visible|enabled|checked|focused|selected|expanded|readonly|required]
```

- Default snapshot returns a flat compact ref list (one line per element, minimal tokens).
- `-i` (interactive-only): omit non-interactive elements. Use this by default.
- `--tree`: return the full nested accessibility tree instead of a flat list.
- `--compact` (with `--tree`): collapse single-child unnamed wrapper nodes.
- `find-element` does a live search without needing a prior snapshot. Id/title/value/class-name/help-text/selected-text patterns are case-insensitive substring matches.
- `get-element` does a live query for current element properties.
- `check-element` returns a boolean check on element state.

## Element interaction

**Requires a snapshot first.** Uses refs from the most recent snapshot.

```
nib click-element <ref> [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib double-click-element <ref> [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib right-click-element <ref> [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib type-element <ref> <text...> [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib focus-element <ref> [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib hover-element <ref> [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
nib scroll-element <ref> <up|down|left|right> [amount] [--window <title>] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
```

`type-element` clicks the element to focus it, then types the text.

## Mouse (coordinate-based fallback)

```
nib mouse-position
nib mouse-move <x> <y>
nib mouse-move-smooth <x> <y>
nib click [--x X --y Y] [--button left|right|middle]
nib double-click [--x X --y Y] [--button left|right|middle]
nib right-click [--x X --y Y]
nib drag <fromX> <fromY> <toX> <toY>
nib scroll <up|down|left|right> [amount]
nib mouse-down [--button left|right|middle]
nib mouse-up [--button left|right|middle]
```

`--x` and `--y` must be provided together or both omitted (uses current position).

## Keyboard

```
nib type <text...>
nib press <keys...>
nib key-down <keys...>
nib key-up <keys...>
nib list-keys
```

`press` simulates pressing keys together (e.g., `nib press leftcmd c` for Cmd+C).
Key names are lowercase. Use `list-keys` to see all available names. Common aliases: enter, space, tab, esc, ctrl, alt, shift, cmd, left, right, up, down, backspace, delete.

## Screen

```
nib screen-size
nib screenshot [--file <path>] [--region x,y,w,h] [--format png|jpg]
nib screenshot-base64 [--region x,y,w,h] [--format png|jpg]
nib color-at <x> <y>
nib highlight <x> <y> <w> <h> [--duration ms] [--opacity 0-1]
nib find-text <text> [--regex] [--language eng,deu,...]
nib read [--region x,y,w,h] [--language eng,...] [--confidence 0-100]
```

- `screenshot` saves to file, `screenshot-base64` returns data URL.
- `find-text` uses OCR to locate text on screen and returns its region.
- `read` extracts text via OCR from the full screen or a region.

## Clipboard

```
nib clipboard-get
nib clipboard-set <text...>
```

## Wait & timing

```
nib wait <ms>
nib wait-for-window [title] [--timeout ms] [--interval ms] [--app <name>] [--pid <id>] [--path <path>] [--bundle-id <id>]
```

`wait-for-window` polls until a matching window appears (default timeout: 10s). Supports window targeting options.

## Batch execution

```
nib batch [file] [--stop-on-error]
```

Executes commands from a `.nib` script file or stdin (one command per line, `#` comments).

## System

```
nib version
nib status
```

---

# Common patterns

## Fill a form and submit

```bash
nib focus-window "My App"
nib snapshot --window "My App" -i
# Parse refs from output, then:
nib type-element @txt:Username "admin" --window "My App"
nib type-element @txt:Password "secret" --window "My App"
nib click-element @btn:Login --window "My App"
```

## Wait for a dialog, then interact

```bash
nib wait-for-window "Confirm" --timeout 5000
nib snapshot --window "Confirm" -i
nib click-element @btn:OK --window "Confirm"
```

## Verify state before acting

```bash
nib check-element @btn:Submit --window "App" --property enabled
# Parse the result field — only proceed if true
nib click-element @btn:Submit --window "App"
```

## Read text from a region

```bash
nib read --region 100,200,400,50
```

## Keyboard shortcuts

```bash
nib focus-window "Editor"
nib press leftcmd a          # Select all
nib press leftcmd c          # Copy
nib press leftcmd v          # Paste
```

## Navigate menus

```bash
nib snapshot --window "App" -i
nib click-element @mbar:File --window "App"
nib wait 300
nib snapshot --window "App" -i    # Re-snapshot to see menu items
nib click-element @mnu:Save --window "App"
```

## Track UI changes after an action

```bash
nib snapshot --window "App"              # baseline
nib click-element @btn:Submit --window "App"
nib wait 500
nib diff --window "App"                  # shows added/removed/changed elements
```

---

# Rules for reliability

1. **Always snapshot before using element refs.** Refs come from the last snapshot — stale refs will fail.
2. **Re-snapshot after UI transitions.** Opening a dialog, navigating pages, or expanding menus changes the element tree.
3. **Use `diff` for incremental updates.** After small changes (typing, toggling), `diff` is faster than a full snapshot.
4. **Use `-i` for snapshots.** Interactive-only mode is faster and produces less noise.
5. **Prefer element refs over coordinates.** Refs survive layout shifts, resolution changes, and minor UI updates.
6. **Use coordinates only as a last resort.** For elements without accessibility info (canvas, custom widgets), fall back to `find-text` + `click --x --y` or OCR-based approaches.
7. **Add short waits after actions that trigger animations or network requests.** Use `nib wait 300` to `nib wait 1000` depending on expected latency.
8. **Check element state before interacting** when the element may be disabled or hidden. Use `check-element` with `--property enabled` or `--property visible`.
9. **Handle errors gracefully.** If a ref is not found, re-snapshot. If a window is not found, use `wait-for-window`. If an action fails, read the error message.
10. **`--window` matches the full title exactly.** Copy titles verbatim from `list-windows` or `snapshot` output. If you only know part of the title, use `--window-contains <text>` (substring) or `--window-regex <pattern>` — but expect an `AMBIGUOUS_WINDOW` error if several windows match, and narrow with `--app` or `--pid`.