Runbook Author

Purpose

Turn an alert definition (a Prometheus rule, a Datadog monitor, a PagerDuty service config) OR a postmortem into a runbook entry the on-call engineer can follow at 3am. Used by the sre agent. The output is a single-page runbook stub the on-call edits as needed — not a final doc, but a structured starting point that prevents the "blank page at 3am" problem.

Scope

• Reads ONE of:
• an alert definition file (Prometheus *.rules.yaml, Datadog

monitor JSON export, generic YAML with name + query)

• a postmortem markdown (postmortems/YYYY-MM-DD-*.md)
• Produces a runbook entry at --out (default

runbooks/<alert-name>.md).

• Does NOT auto-publish to a wiki; the operator commits or pastes

the file themselves.

• Does NOT execute any of the actions it suggests — the runbook is

text, not a script.

When to use

• A new alert was added — runbook entry should land in the same PR.
• A postmortem identified a missing runbook (action item: "write

runbook for the disk-full alert").

• During on-call rotation handoff, to fill in stub runbooks for

alerts that exist but lack docs.

When NOT to use

• For an alert whose root cause is genuinely unknown. "Investigate"

is not a first action. If the team doesn't know the symptom-to- cause mapping yet, run a game-day or shadow an incident first.

• For automation / auto-remediation. Runbooks are human-targeted;

if every step is a script, the work belongs in code, not docs.

• As a substitute for SLO definition. A runbook tells the on-call

what to do; the SLO tells them whether to wake up.

Automated pass

1. Detect input mode:

sh if [ -n "${ALERT:-}" ]; then mode=alert elif [ -n "${POSTMORTEM:-}" ]; then mode=postmortem else echo "pass --alert or --postmortem" >&2; exit 2 fi

1. Extract the salient fields per mode:

alert mode — pull name, expr/query, for, severity, summary annotation, runbook_url annotation if already set. sh yq '.groups[].rules[] | select(.alert)' "$ALERT"

postmortem mode — pull title, root cause section, detection section, mitigation steps, action items.

1. Render the runbook template. The skill enforces the section

order — the on-call's eye expects the same order on every entry:

markdown # Runbook: <alert name>

Severity: sev-1 | sev-2 | sev-3 Owner team: <team> Pages: yes | no SLO impact: <which SLO budget this burns>

## Symptom What the alert means in one sentence. What the user sees, not the metric.

## First 5 actions

1. <single concrete check or action>
2. <next>
3. <next>
4. <next>
5. <next>

## If the first 5 don't help Escalate to <secondary on-call / team lead>. Page <service-owner> if the symptom is still active after <N> minutes.

## Known false positives

• <condition under which this fires but isn't real>

## Related

• Dashboard: <link>
• SLO: <link>
• Recent incidents: <links to postmortems>

1. For alert-mode, populate the Symptom section from the alert's

summary annotation; populate First 5 actions with stubs:

1. confirm the alert by checking the dashboard at <link>
2. check recent deploys (kubectl rollout history / equivalent)
3. check the dependency graph for upstream alerts firing simultaneously
4. check error logs for the affected service in the last 15m
5. if customer-facing, post to #incidents and start an incident channel

These are STUBS — the operator overwrites them with the real symptom-specific actions. The skill marks them with a <!-- TODO: replace --> comment to make that obvious.

1. For postmortem-mode, populate First 5 actions from the

postmortem's mitigation section. If the postmortem has 8 mitigation steps, take the first 5 and list the rest under "If the first 5 don't help."

1. Write the file at --out (or

runbooks/<slug-of-alert-name>.md). Refuse to overwrite if the path exists; require --force.

Manual pass

For a one-off entry, the operator copies the template above into a new file and fills it in. Five sections, ordered. The point of the skill is consistency across runbooks, not novel content per entry.

Known gotchas

• Stub actions get committed verbatim. The biggest failure mode

is the operator commits the skill output without replacing the TODOs. The skill emits the file with a banner at the top: > [!WARNING] This runbook contains TODO stubs. Replace before on-call relies on it. The CI lint step in the runbooks repository should reject files with <!-- TODO: replace --> still present.

• Runbook URL not wired into the alert. The runbook lives in

the repo, but the alert needs annotations.runbook_url pointing at the rendered page (e.g., a generated GitHub Pages URL). The skill does NOT update the alert def — that's a separate edit. Easy to forget; the runbook is useless if the on-call can't find it from the page.

• First 5 actions get long. The "5" is a forcing function.

If the on-call needs 12 steps, the alert is too coarse — split it into multiple alerts each with a tighter symptom. Resist the urge to bullet-list 14 items.

• Postmortem-derived runbooks go stale. A runbook generated

from a 2024 postmortem describes the 2024 architecture. Re- validate runbooks during on-call shadowing; mark stale ones with a Last reviewed: trailer.

• Pages: yes for everything. Reviewing the corpus of runbooks,

if every alert pages, the team is at burnout risk. The skill does not enforce this — it's a culture call — but the runbook's Pages: field surfaces the count.

References

• Google SRE Book chapters 11 (on-call) and 12 (effective

troubleshooting).

• lib/skills/postmortem-write/SKILL.md — companion: produces

the postmortem this skill can read.

• runbooks/ — project-level runbook directory.
• The alerting repo's CI lint that catches unreplaced TODO stubs.