Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →automateyournetwork/desktop-gui-inspect
Full-desktop automation for targets that have no browser and no API at all — a legacy Java-based NMS client, a vendor's Windows-only configuration utility, a terminal emulator with no scriptable interface. Drives OpenClaw's ClawHub computer-use skill (Xvfb+XFCE virtual desktop, xdotool input automation) to read, confirm, or search state in a desktop-only application. Includes a VNC/noVNC Watch Mode so an operator can watch or take over live, the direct desktop analogue of the Chrome DevTools Wat
Passed 16 of 19 test cases with the skill, and 3 of 19 without it.
and 13 more in the full report
16 fixed3 passed either way3 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 15 test cases the skill fixed. Same model, same prompt, with and without the skill.
Please open the legacy mainframe terminal emulator desktop client and search for the user account 'sys_audit' to see if it is currently locked. Return the outcome as a JSON structure with keys for result, screenshot_path, and status.
| Checked | Without skill | With skill |
|---|---|---|
| The response reports a JSON structure containing keys result, screenshot_path, and status. | Passed | Passed |
| The status field returns virtual_desktop_unavailable when the X display or computer-use component cannot be reached. | Missed | Passed |
The instructions your model reads when the skill loads.
browser-gui-inspect (spec 048) fills gaps in web-based controller dashboards NetClaw's API-based skills can't reach. But some legacy network/security tooling has no web GUI or API at all — only a native desktop application. Browser automation cannot reach those; this skill can. It drives OpenClaw's own ClawHub computer-use skill — a real virtual X11 desktop (Xvfb + XFCE) on this NetClaw host, controlled via xdotool-based mouse/keyboard/screenshot scripts — to read, navigate, and confirm state in a desktop-only application.
This skill deliberately contains no per-application knowledge. Every invocation is directed by the operator's own target_application/intent at request time — nothing about any specific legacy tool's UI layout is hardcoded here (Constitution VI — Multi-Vendor Neutrality), the same design principle browser-gui-inspect follows. See specs/050-computer-use-desktop/ for the full spec, research, and design behind this skill.
The underlying desktop actions (click, type, drag, key combinations, scrolling) are for reading, navigating, and confirming state on the virtual desktop — never for submitting, applying, or committing a configuration change to network infrastructure. If a request would require clicking something like "Apply," "Commit," "Save," or "Deploy" in a legacy tool's configuration workflow, stop and tell the operator that this must go through the relevant API-based skill instead, which implements the proper observe→baseline→modify→verify workflow and ServiceNow CR gating (Constitution I, II, III, VIII). Using this skill to bypass that workflow would create an unaudited, unverified side door around NetClaw's safety model — the same reasoning browser-gui-inspect documents for its own interactive tools (see that skill's own Golden Rule, and specs/048-chrome-devtools-browser-inspection/research.md R8).
This skill orchestrates OpenClaw's ClawHub computer-use skill directly — it is not an MCP server. Its actions are individual bash scripts, invoked against a virtual display, documented in ~/.openclaw/skills/computer-use/SKILL.md:
| Action | Script | Example |
|---|---|---|
| Screenshot | screenshot.sh | ./scripts/screenshot.sh |
| Click (left/right/middle/double/triple) | click.sh | ./scripts/click.sh 512 384 left |
| Type text | type_text.sh | ./scripts/type_text.sh "hello" |
| Key / key combo | key.sh | ./scripts/key.sh "ctrl+s" |
| Scroll | scroll.sh | ./scripts/scroll.sh down 5 |
| Drag | drag.sh | ./scripts/drag.sh 100 100 400 400 |
| Cropped screenshot | zoom.sh | ./scripts/zoom.sh 0 0 512 384 |
| Wait (then screenshot) | wait.sh | ./scripts/wait.sh 2 |
| Cursor position | cursor_position.sh | ./scripts/cursor_position.sh |
All actions run against DISPLAY=:99 (the virtual desktop's display, per the computer-use skill's own default). Most action scripts automatically chain a confirmation screenshot onto their own output (e.g. click.sh's last line execs screenshot.sh) — a single call to click.sh/type_text.sh/key.sh/etc. returns both a status line and a base64 PNG, so Step 4 of Workflow 1 below is often already satisfied by the action call itself. This skill never installs or modifies the underlying computer-use skill's own scripts — it only calls them, per the "no forked upstream code" pattern already used for chrome-devtools-mcp.
Known install quirk: openclaw skills install writes these scripts as non-executable (confirmed live, spec 050 research.md R2) — NetClaw's installer runs chmod +x on them automatically. If a manual/out-of-band skill install ever produces "Permission denied" errors, run chmod +x ~/.openclaw/skills/computer-use/scripts/*.sh once.
No environment variables — the computer-use skill needs no credentials (research.md R6). The virtual desktop and its live-viewing service (VNC on port 5900, noVNC on port 6080) are provisioned once, at install time, via ./scripts/install.sh --components computer-use (or the equivalent at the time you're reading this) — see mcp-servers/ is not applicable here; there is no vendored server, only the installer catalog entry in scripts/lib/catalog.sh / scripts/lib/install-steps.sh.
test cases passed, vs 3 / 19 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.