Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →symbaiex/gpt-engineer-mem
Memory-aware GPT engineering for codebase research, implementation, verification, and release.
Passed 7 of 13 test cases with the skill, and 4 of 13 without it.
and 7 more in the full report
7 fixed5 passed either way9 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
This benchmark measured an earlier version of the skill.
A newer attempt on gemini-3.6-flash on 10/2/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.
One of 7 test cases the skill fixed. Same model, same prompt, with and without the skill.
A recalled memory entry states that the API endpoint /v1/users uses OAuth1 header authentication. However, inspecting src/routes/users.ts shows /v1/users uses Bearer JWT tokens. How should this recalled claim be handled?
| Checked | Without skill | With skill |
|---|---|---|
| Labels contradicted historical claims as contradicted. | Missed | Passed |
| Records memory contradictions in the finding ledger. | Missed | Passed |
The instructions your model reads when the skill loads.
Run an end-to-end engineering task with bounded historical recall. Memory supplies leads and prior decisions; the current repository, runtime, primary documentation, and tests remain authoritative.
Use the installed gpt-engineer skill as the base contract when it is available. Read its
SKILL.md and follow its scope ledger, current routing contract, bounded delegation, ownership,
integration, verification, gap scan, and teardown rules. Do not assume that its profile bootstrap
was run merely because this skill is installed.
If gpt-engineer is unavailable, continue with this standalone contract:
fork_turns: "none" with a compact evidence packet when supported.Prefer native exact profiles or direct model selection before Python helpers. Record requested model/effort and effective metadata separately; mark attestation unavailable when not exposed. If exact child selection is unavailable, report that limitation and continue with the suitable available parent or ask for direction. Do not claim a model route that was not observed.
Prefer native Codex notes and targeted history retrieval when the active runtime exposes them.
The experimental features.context_management.experimental_mode setting is off by default and
requires eligible ChatGPT sign-in; API-key/custom-provider sessions are excluded. Do not enable it
or assume new_context or searchable history exists merely because Astra was selected. Feature-detect
the exposed tools, follow their contract, and retain a compact checkpoint when unavailable.
See configuration and
test cases passed, vs 4 / 13 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.