Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →borghei/computer-use-automation
This skill should be used when the user asks to "build a computer-use agent", "automate a GUI with an AI agent", "when to use computer use vs an API", "make browser automation reliable", or "design screenshot-driven agent actions".
Passed 17 of 21 test cases with the skill, and 11 of 21 without it.
and 5 more in the full report
7 fixed11 passed either way4 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.
Just draft it: I have a local file named `plan.json` containing planned GUI actions. What command line invocation scans this file using the action safety linter?
| Checked | Without skill | With skill |
|---|---|---|
| Includes the --file plan.json argument when running the linter script | Missed | Passed |
The instructions your model reads when the skill loads.
Category: Engineering Domain: AI Agents
The Computer Use Automation skill helps you design AI agents that operate a graphical interface the way a person does — take a screenshot, reason about what is on screen, then click, type, scroll, or navigate, and repeat. It covers the core perception→reason→action loop, the decision of when computer-use is the right tool versus a structured API/MCP tool (prefer a real API whenever one exists; reach for computer-use only for GUIs with no programmatic surface), reliability patterns (grounding every action in the current screenshot, verifying after each step, recovering from misclicks), safety guardrails (confirmation gates for destructive actions, sandboxing, avoiding blocking dialogs), and how to evaluate a computer-use agent. It is model-agnostic — the patterns apply to any computer-use-capable model and any GUI tool surface.
Before designing or auditing a computer-use agent, confirm these inputs. If any is unknown or vague, ASK — do not assume:
tool_choice_advisor.py vs action_safety_linter.py)Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.
# Decide computer-use vs API/MCP for a target
python scripts/tool_choice_advisor.py --api-exists no --gui-stability high --volume low --json
# Lint a planned action sequence for safety/reliability gaps
python scripts/action_safety_linter.py --file planned_actions.json
# Read actions from stdin and emit a markdown risk report
echo '[{"type":"click","target":"Delete"},{"type":"submit","target":"Confirm"}]' \
| python scripts/action_safety_linter.py --format markdown
| Tool | Purpose | Key Flags |
|---|---|---|
tool_choice_advisor.py | Recommend computer-use vs structured API/MCP for a target, with rationale | --api-exists, --gui-stability, --volume, --reversible, --json |
action_safety_linter.py | Scan a planned action list for destructive verbs, missing verification, missing confirmation gates, and dialog-triggering patterns | --file, --format, --json |
test cases passed, vs 11 / 21 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.