Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

CAO Provider Creator · awslabs | DecimalAI

agent toolingby awslabs

CAO Provider Creator

awslabs/cao-provider

Source awslabs/cli-agent-orchestrator ↗License Apache-2.0

Create a new CLI agent provider for CAO (CLI Agent Orchestrator).

SkillScore86Maintained

Automatic

AI Engineer pack

Last verified 8/14/2026

What it changes

Passed 18 of 21 test cases with the skill, and 7 of 21 without it.

Without skill32%

With skill86%

Checks it passed that the model alone missed

Specifies using mock objects or patches for the tmux client
Lists WAITING_USER_ANSWER as the first status state checked in precedence
Recommends defining regex patterns at module level rather than inside methods

and 13 more in the full report

12 fixed7 passed either way3 still failing0 broke

Pass-rate lift: +55 pp
Tokens: +48%
Agent turns: 0%
Response time: -45%

Model gemini-3.6-flashMeasured 8/14/2026v1 · latest is v219 of 22 cases passed with skill · 21 comparable

Tokens, turns and time describe resource use, not a dollar price.

This benchmark measured an earlier version of the skill.

A newer attempt on gemini-3.6-flash on 10/3/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/awslabs-cao-provider/SKILL.md https://app.decimal.ai/s/awslabs-cao-provider/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add awslabs-cao-provider, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/awslabs-cao-provider/ and pins it in .decimal/skills.lock

decimalai skills add awslabs-cao-provider

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/awslabs-cao-provider/SKILL.md https://app.decimal.ai/s/awslabs-cao-provider/SKILL.md

Raw file: /s/awslabs-cao-provider/SKILL.md ↗ · machine-readable summary at /s/awslabs-cao-provider.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 11

Prompt

Which file and data structure must be modified to ensure CAO prompts for workspace permissions before launching a provider that reads and writes local disk files?

What was checked, without and with the skill
Checked	Without skill	With skill
Identifies adding the provider name to PROVIDERS_REQUIRING_WORKSPACE_ACCESS in src/cli_agent_orchestrator/cli/commands/launch.py	Missed	Passed

case-14 · gemini-3.6-flash · 8/14/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md173 lines4 files

CAO Provider Creator

Guide for creating a new CLI agent provider for CLI Agent Orchestrator. A "provider" is an adapter that lets CAO interact with a specific CLI-based AI agent through tmux.

What You're Building

A provider translates between CAO's unified interface and a specific CLI tool's terminal output. It needs to:

Launch the CLI tool in a tmux window with the right flags
Detect status by parsing terminal output (IDLE, PROCESSING, COMPLETED, ERROR, WAITING_USER_ANSWER)
Extract responses from the terminal buffer after the agent finishes
Clean up when the terminal is deleted

Before You Start

Gather this information about the target CLI:

What command launches it? (e.g., claude, kiro-cli chat, codex)
What does the idle prompt look like? (e.g., > , ❯ , ask a question)
What does the processing state look like? (e.g., spinner characters, "Thinking...")
How are responses formatted? (e.g., preceded by ⏺, inside a box, plain text)
Does it support --dangerously-skip-permissions or similar flags?
Does it have a REPL mode or is it single-shot?
How does it handle MCP servers? (CLI flags, config file, agent JSON)
Does it use alt-screen (full-screen TUI) or scrollback (inline output)? This fundamentally changes status detection logic — see lesson #16
What's the exit command? (/exit, /quit, Ctrl+C)

Step-by-Step Implementation

Step 1: Add to ProviderType enum

File: src/cli_agent_orchestrator/models/provider.py

class ProviderType(str, Enum):
    # ... existing providers ...
    NEW_CLI = "new_cli"

The value string is used everywhere — in API requests, database, config. Use snake_case.

Track record

Benchmark

18 / 21

test cases passed, vs 7 / 21 without

Adoption

Users: 0
Times used: 0

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v5 — 10/1/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

awslabs / cli-agent-orchestrator

skills/cao-provider/SKILL.md

View on GitHub ↗

License Apache-2.0Imported 10/1/2026Version v2Bundled 3 files

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

CAO Provider Creator

What You're Building

Before You Start

Step-by-Step Implementation

Step 1: Add to ProviderType enum

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills