[omh] Hermes UltraQA workflow: adversarial QA and fix loops.
SkillScore71Maintained
Automatic
◆QA & Test Engineering pack
Last verified 8/27/2026
What it changes
Passed 17 of 24 test cases with the skill, and 6 of 24 without it.
Without skill25%
With skill71%
Checks it passed that the model alone missed
States the change of scope to the export API.
Routes the task back to planning when expected behavior is unclear.
Recommends using the smallest verification that can prove the claim.
and 9 more in the full report
11 fixed6 passed either way7 still failing0 broke
Pass-rate lift
+46 pp
Tokens
+26%
Agent turns
0%
Response time
-13%
Model gemini-3.6-flashMeasured 8/27/2026v417 of 24 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 11
Prompt
We need you to implement the actual Python function for our new token bucket rate limiter in rate_limiter.py. Please write the complete production implementation code right now.
What was checked, without and with the skill
Checked
Without skill
With skill
Declines feature code implementation, identifying it as an authoring task outside QA verification scope.
Missed
Passed
case-05 · gemini-3.6-flash · 8/27/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md109 linessingle file
Ultraqa
This is a Hermes-native ultraqa workflow skill.
Why This Exists
ultraqa exists to keep verification work explicit, evidence-backed, and inside the Hermes/executor boundary instead of relying on ad hoc chat narration.
Do Not Use When
The request is casual chat, a status-only acknowledgement, or another workflow has stronger routing evidence.
The user needs implementation, review, CI, merge, or external publishing evidence that has not been delegated or observed.
Examples
Good example:
Prompt: $ultraqa test the setup wizard with hostile install paths, stale config, and missing PATH cases.