Authoring Signals scouts

A scout is a scheduled agent that wakes on its own interval, looks at one PostHog project, decides what's genuinely worth surfacing, and emits it as a finding into the Signals inbox — or closes out empty, which is a real outcome. PostHog ships a fleet of canonical scouts (a cross-product generalist plus per-surface specialists). This skill helps you and your agent adapt those canonical scouts to a specific project, or author new scouts from scratch for a use case the fleet doesn't cover.

A scout is just an LLMSkill whose name starts with signals-scout-. The harness discovers scouts by globbing signals-scout-* over the project's skills, loads the body verbatim as the agent's system prompt, and progressively reads any bundled reference files on demand. The signals-scout- name prefix is load-bearing: a skill named anything else will never run as a scout.

The job before the writing

Don't write a scout in the abstract. Ground it in the target project first — a scout is only as good as its fit to the data it watches.

1. Read the project. posthog:signals-scout-project-profile-get returns the

deterministic snapshot the scout itself cold-starts from: products in use, top events with reach/burst metrics, integrations, existing inbox counts. If the scout watches a specific event, confirm it exists and check its shape with posthog:read-data-schema. A scout for an event the project doesn't capture is dead on arrival.

1. See what already runs. posthog:signals-scout-config-list lists every existing

scout on the project with its schedule, enabled, and emit posture. Don't duplicate a surface a canonical scout already covers — adapt that one instead.

1. Read the closest canonical scout. It's your template and your reference shape. Pull

it with posthog:llma-skill-get {"skill_name": "signals-scout-<x>"} (per-team rows) or read it from the repo at products/signals/skills/signals-scout-*/. The generalist (signals-scout-general) is the broad template; pick a specialist (-error-tracking, -ai-observability, -logs, -revenue-analytics, -surveys, -csp-violations, -observability-gaps) if your scope is domain-tight.

1. Skim the inbox. posthog:inbox-reports-list shows what findings are actually

landing — calibrate so your scout adds signal, not noise.

Choose the path

There are two independent decisions: what you're building, and where it lives.

What

| Situation | Approach | | ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | A canonical scout is close but too broad / too noisy / missing a disqualifier for this project | Adapt it — narrow the scope, add disqualifiers, retune thresholds. | | You want a surface no canonical scout covers (a custom event, a product-specific funnel) | New scout from scratch — copy the closest canonical scout as scaffolding, replace the domain discriminator + explore patterns. | | You only want to change _when_ / _whether_ a scout runs | No authoring — just tune the config (see Run posture). |

Where

| Path | Mechanism | Use when | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- | | Per-team (the common user path) | Create/edit a signals-scout-* LLMSkill row in the project's skills store via posthog:llma-skill-create / -update / -file-create. | Customizing for one project. The harness globs the row in on the next tick; canonical sync leaves your edited ("diverged") row alone. | | Canonical (PostHog contributors) | Edit disk under products/signals/skills/signals-scout-*/, lint/build, open a PR. | Improving a scout for _every_ enrolled project. lazy_seed mirrors it onto all enrolled teams on the next tick. |

Adapting-in-place tradeoff: editing a canonical scout's row for your team marks it diverged — you stop receiving upstream improvements to that scout. If you only need an _additional_ behavior, prefer authoring a new, differently-named scout (signals-scout-<your-scope>) and leaving the canonical one intact.

See references/lifecycle-and-testing.md for the exact skills-store calls, the build/lint commands, and how seeding works.

Write the scout

Follow references/scout-anatomy.md — it has the frontmatter schema, the canonical body structure (quick close-out → orient → domain discriminator → explore patterns → save-memory → decide → disqualifiers → close-out), the lean-body rule, and copy-ready skeleton templates for both a specialist and the generalist.

Two craft references the whole fleet reasons in terms of — a good scout's Decide and memory sections are built on them, so read them before writing those sections:

• references/emit-contract.md — what emit-signal takes,

the weight vs. confidence rubrics, severity, dedupe keys, finding_id, the description prose contract, and a worked example. This is how your scout decides _what clears the bar_ and _how to write the finding_.

• references/dedupe-and-memory.md — the four-states

classifier (net-new / material-update / already-covered / addressed-or-noise), the scratchpad key-prefix vocabulary, and the cross-project noise patterns. This is how your scout avoids re-emitting and learns across runs.

The single most important design decision in any scout is its signal-vs-noise discriminator — the cheap profile-shape read that separates "worth investigating" from "baseline". For error tracking it's the count vs distinct_users ratio; for CSP it's reach over raw count. Your new scout needs its own. Name it explicitly near the top of the body so every run anchors on it.

Run posture (config)

A scout's schedule and emit behavior live on its SignalScoutConfig, separate from the skill body. Tune with posthog:signals-scout-config-update (find the id via -config-list):

• run_interval_minutes — 10 to 43200. Default 60 (hourly). Slow a chatty or expensive

scout by raising this.

• enabled — false pauses the scout entirely (coordinator skips it).
• emit — false = dry-run: the scout runs and logs its reasoning but writes nothing

to the inbox. New and freshly-edited scouts should run dry-run first.

Test loop

You can't force a synchronous run as a user — scouts fire on their schedule. The feedback loop is dry-run + inspect:

1. Ship the scout with emit=false and a short run_interval_minutes so it fires soon.
2. After a tick, read what it did: posthog:signals-scout-runs-list (run summaries),

-runs-retrieve (full reasoning for one run), and -scratchpad-search (the durable memory it wrote). In dry-run, runs show what it _would_ have emitted.

1. Refine the body — tighten the discriminator, add disqualifiers for whatever it

false-positived on, fix the emit calibration.

1. When the dry-run findings look right, flip emit=true. Restore the interval to

something sustainable (hourly+).

Repo contributors get a faster loop — hogli sync:skill and the harness's local run path; see references/lifecycle-and-testing.md.

To read what your scouts are doing rather than change them — surveying the fleet, inspecting individual runs, the scratchpad memory, and assessing performance — use the read-only companion skill exploring-signals-scouts. Keep the two in sync when the scout config / run / scratchpad surfaces change.

Quality bar for a v1 scout

• A named, cheap signal-vs-noise discriminator anchored near the top.
• A quick close-out so a quiet run is cheap (don't pay for deep exploration when the

watched surface is at baseline or absent).

• 2–4 concrete explore patterns with the actual queries/tools to run — starting

points, not a rigid checklist.

• Disqualifiers listing this project's known noise (single-user quirks, dev-env

bursts, allowlisted entities).

• A Decide section calibrated against the emit contract (confidence ≥ 0.65 to emit;

below that, write memory).

• Save-memory guidance using the scratchpad prefixes so the scout gets smarter each run.
• A lean body (push depth into references/) — every line is a recurring token cost on

every run.