Passed 16 of 21 test cases with the skill, and 12 of 21 without it.
Without skill59%
With skill77%
Checks it passed that the model alone missed
The issue body includes a GAIT session ID reference.
The pull request link is added to the GAIT audit trail.
The response specifies GITHUB_PERSONAL_ACCESS_TOKEN as the environment variable name.
and 1 more in the full report
4 fixed13 passed either way5 still failing0 broke
Pass-rate lift
+18 pp
Tokens
-14%
Agent turns
0%
Response time
-22%
Model gemini-3.6-flashMeasured 8/16/2026v117 of 22 cases passed with skill · 21 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 3
Prompt
We are configuring automated scripts to interact with our GitHub organization for network repository operations. What environment variable name should be set to authenticate these API interactions with repository access permissions?
What was checked, without and with the skill
Checked
Without skill
With skill
The response specifies GITHUB_PERSONAL_ACCESS_TOKEN as the environment variable name.
Missed
Passed
case-04 · gemini-3.6-flash · 8/16/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md55 linessingle file
GitHub Operations Skill
Available Capabilities
Repository Operations
Search repositories, list branches, get file contents
Create/update files, create branches
Compare branches, list commits
Issues
Create, list, search, update, and comment on issues
Use issues to track network findings, audit results, and remediation tasks
Pull Requests
Create PRs for config changes, list PRs, get PR details
Review and merge PRs, add comments
Code Search
Search across repositories for configuration patterns
Find references to specific IPs, VLANs, ACLs in config repos
GitHub Actions
List workflow runs, get workflow status
Trigger workflows for automated testing
Workflow Patterns
Network Finding → GitHub Issue
When you discover a network problem (via pyATS, health check, etc.):
Create a GitHub issue with the finding details
Include device name, symptom, recommended fix
Label appropriately (bug, security, performance)
Reference the GAIT session ID for audit trail
Config Change → Pull Request
When a config change is approved:
Create a branch from main
Commit the new/updated config file
Open a PR with change details and ServiceNow CR number