Docs reliability review

Follows the written setup path and reports where the docs drift from reality.

Trigger

Use when the user wants to know whether the repo documentation is actually trustworthy for an agent starting fresh.

Workflow

1. If a compatibility scan result is already available from the parent task, use it as context. Otherwise run the compatibility scan once.
2. Read the obvious documentation surfaces: README, setup docs, env docs, and contribution or agent guidance.
3. Follow the documented setup and run path as literally as practical.
4. Note where docs are accurate, stale, incomplete, or misleading.
5. Pick a specific score instead of a round bucket. Start from these anchors and move a few points if the evidence clearly warrants it:

• around 93/100 if the docs lead to the working path with little or no correction.
• around 84/100 if the docs drift in places but an agent can still get to the right setup or run path without much guesswork.
• around 68/100 if the docs are stale enough that the agent has to reconstruct important steps from the tree or CI.
• around 27/100 if the docs point the agent down the wrong path or omit key steps you need to proceed.
• around 12/100 if the real path depends on private docs or internal context that is not available in the repo.

1. Prefer a specific score such as 81, 85, or 92 over a multiple of ten when that is the more honest read.

Output

Reply in plain text only (no markdown fences, no # headings, no emphasis syntax). Use this layout:

First line: Docs Reliability Score: <score>/100

Then a short summary paragraph.

Then the line Problems followed by one bullet per line using - .

• Base the score on what happened when you followed the docs.
• Build Problems from real mismatches, omissions, or misleading guidance.
• If the repo is blocked on secrets or infrastructure, say so plainly and still use the same output shape.
• Minor drift or stale references should not drag a good repo into the mid-60s if the real path is still easy to recover.
• Score the damage from the drift, not the mere existence of drift.