Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Apply Fly.io security best practices for secrets management, private networking, TLS certificates, and deploy token scoping. Trigger: "fly.io security", "fly secrets", "fly.io TLS", "fly.io private network".
.claude/skills/jeremylongshore-flyio-security-basics/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-05 | ✗→✓ | ▲ Improved | 27% | 0% |
| case-09 | ✗→✓ | ▲ Improved | -8% | 0% |
| case-14 | ✗→✓ | ▲ Improved | 25% | 0% |
| case-21 | ✗→✓ | ▲ Improved | 18% | 0% |
| case-01 | ✓→✓ | = Same ✓ | 48% | 0% |
Protect the control plane, deployed code, runtime secrets, images, network paths, and support evidence as one system. Scoped tokens reduce control-plane authority, but deploy access remains highly sensitive because new code can read secrets injected into Machines.
Distinguish interactive operators, app deploy automation, organization automation, read-only monitoring, SSH, Machine-exec, WireGuard, database, and external-service identities.
Use app deploy tokens for one app, read-only organization tokens for observation, and short-lived command or SSH tokens for bounded access. Review and revoke by token ID.
Store app values through Fly secrets, inspect only names and digests, stage changes when appropriate, and prevent deployed code, logs, crashes, and support bundles from exfiltrating values.
Require reviewed immutable images, dependency and vulnerability gates, provenance, protected production environments, release approval, and rollback. Treat image registry and build credentials separately.
Map public services, certificates, 6PN, Flycast, WireGuard peers, outbound destinations, and application authentication. Private reachability does not imply trusted requests.
Exercise token rotation, emergency revocation, secret update, deploy rollback, access review, log redaction, and incident escalation with timestamped receipts.
Current guidance deprecates routine use of the all-powerful fly auth token output. Store scoped tokens in an approved secret manager and expose them only to the intended process. Never place tokens, secret values, WireGuard keys, or database URLs in source or evidence.
Use Read and Grep to inspect application configuration, deployment evidence, provider documentation, fixtures, logs, schemas, and existing tests before proposing a change. Use Write or Edit only for an approved plan, configuration, implementation, test, or redacted receipt. Do not create, deploy, scale, restart, stop, suspend, destroy, rotate, revoke, expose, or migrate live Fly.io resources without explicit operator approval.
Return the target organization, app, environment, region set, Machine or database identifiers, source-contract fingerprint, evidence, unresolved risks, rollback state, and final decision without exposing tokens, secrets, connection strings, or customer data.
A production app has one expiring app deploy token, a separate read-only monitoring token, short-lived Machine-exec access for migrations, protected image promotion, named WireGuard peers, and a quarterly drill that revokes and replaces each automation credential.
| Failure | Response | | --- | --- | | Unknown or ownerless token | Revoke after dependency review or assign owner and expiry immediately; do not leave indefinite authority. | | Secret appears in logs or evidence | Contain access, rotate the secret, sanitize retained artifacts, and investigate deployed code and pipeline output. | | Private service lacks app authentication | Add workload authentication and authorization; 6PN reachability is not an identity decision. |
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | pass→pass | 16,931 | 15,324 | -9% | 1 | 1 | 0% | 1,988 | 2,952 | +48% | 0 | 0 | — |
case-02 | pass→pass | 14,707 | 13,384 | -9% | 1 | 1 | 0% | 1,636 | 2,680 | +64% | 0 | 0 | — |
case-03 | pass→pass | 21,312 | 18,988 | -11% | 1 | 1 | 0% | 2,482 | 3,518 | +42% | 0 | 0 | — |
case-04 | pass→pass | 16,424 | 13,838 | -16% | 1 | 1 | 0% | 1,866 | 2,771 | +48% | 0 | 0 | — |
case-05 | fail→pass | 12,957 | 7,604 | -41% | 1 | 1 | 0% | 1,344 | 1,705 | +27% | 0 | 0 | — |
case-06 | pass→pass | 17,774 | 14,887 | -16% | 1 | 1 | 0% | 2,246 | 3,020 | +34% | 0 | 0 | — |
case-07 | pass→pass | 15,658 | 13,268 | -15% | 1 | 1 | 0% | 1,901 | 2,848 | +50% | 0 | 0 | — |
case-08 | pass→pass | 21,750 | 18,386 | -15% | 1 | 1 | 0% | 2,578 | 3,294 | +28% | 0 | 0 | — |
case-09 | fail→pass | 20,371 | 10,197 | -50% | 1 | 1 | 0% | 2,434 | 2,245 | -8% | 0 | 0 | — |
case-10 | pass→pass | 14,381 | 13,006 | -10% | 1 | 1 | 0% | 1,603 | 2,565 | +60% | 0 | 0 | — |
case-11 | pass→pass | 8,978 | 12,455 | +39% | 1 | 1 | 0% | 1,501 | 2,552 | +70% | 0 | 0 | — |
case-12 | pass→pass | 12,606 | 10,697 | -15% | 1 | 1 | 0% | 1,131 | 2,158 | +91% | 0 | 0 | — |
case-13 | pass→pass | 12,866 | 11,960 | -7% | 1 | 1 | 0% | 1,314 | 2,391 | +82% | 0 | 0 | — |
case-14 | fail→pass | 25,620 | 21,139 | -17% | 1 | 1 | 0% | 3,318 | 4,150 | +25% | 0 | 0 | — |
case-15 | pass→pass | 18,937 | 18,265 | -4% | 1 | 1 | 0% | 2,147 | 3,445 | +60% | 0 | 0 | — |
case-16 | pass→pass | 22,806 | 21,780 | -4% | 1 | 1 | 0% | 2,849 | 4,185 | +47% | 0 | 0 | — |
case-17 | pass→pass | 17,134 | 7,822 | -54% | 1 | 1 | 0% | 2,376 | 1,792 | -25% | 0 | 0 | — |
case-18 | pass→pass | 14,503 | 10,755 | -26% | 1 | 1 | 0% | 1,704 | 3,106 | +82% | 0 | 0 | — |
case-19 | pass→pass | 16,992 | 19,374 | +14% | 1 | 1 | 0% | 2,102 | 3,732 | +78% | 0 | 0 | — |
case-20 | pass→pass | 11,944 | 14,587 | +22% | 1 | 1 | 0% | 1,230 | 2,972 | +142% | 0 | 0 | — |
case-21 | fail→pass | 15,862 | 10,479 | -34% | 1 | 1 | 0% | 1,835 | 2,174 | +18% | 0 | 0 | — |
case-22 | pass→pass | 22,506 | 15,085 | -33% | 1 | 1 | 0% | 2,844 | 2,984 | +5% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +18 percentage points is the difference between those two pass rates over the 22 comparable cases.
The publisher has shipped newer versions since this run, so these numbers describe v2, not the version currently listed.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
| Model | Method | Date | Lift |
|---|---|---|---|
| gemini-3.6-flash | verified | 8/30/2026 | +21% |
Other measured skills in the registry, with their headline benchmark lift.