Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

Prompt Cache Optimizer Skill · miosa-osa | DecimalAI

miosa-osa/prompt-cache-optimizer

NewSkillScoreMaintained

Optimize token usage through prompt caching and compression

AI/MLby miosa-osa

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

AI Engineer pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/miosa-osa-prompt-cache-optimizer/SKILL.md

decimalai skills pull miosa-osa-prompt-cache-optimizer --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/miosa-osa-prompt-cache-optimizer/SKILL.md https://app.decimal.ai/s/miosa-osa-prompt-cache-optimizer/SKILL.md

Raw file: /s/miosa-osa-prompt-cache-optimizer/SKILL.md ↗ · machine-readable summary at /s/miosa-osa-prompt-cache-optimizer.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 46h ago

Yes — +36% lift-6% tokens0% turns82% of whole cases pass with skill (18/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-15	✗→✓	▲ Improved	-22%	0%
case-02	✗→✓	▲ Improved	32%	0%
case-07	✗→✓	▲ Improved	-28%	0%
case-08	✗→✓	▲ Improved	-30%	0%
case-09	✗→✓	▲ Improved	-18%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/3/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

miosa-osa / canopy

library/skills/ai-patterns/prompt-cache-optimizer/SKILL.md

View on GitHub ↗

Imported 8/3/2026Version v1

Skill contents

SKILL.md

Prompt Cache Optimizer Skill

Reduces token costs by 50-90% through intelligent caching and compression.

When to Activate

Large context windows (>50K tokens)
Repeated similar queries
Long-running sessions
Cost-conscious operations

Optimization Layers

Layer 1: Semantic Caching

Query → Embedding → Similarity Search → Cache Hit/Miss
         ↓              ↓
    Vector Store    Return cached or call LLM

Cache hits provide 100% token savings with near-instant response.

Layer 2: Prompt Compression (LLMLingua-2)

Light: 2-3x reduction, <5% accuracy impact
Moderate: 5-7x reduction, 5-15% accuracy impact
Aggressive: 10-20x reduction, requires validation

Layer 3: Strategic Context Placement

Mitigate "lost in the middle" problem:

Place most important information at START and END
Middle content has 30-50% lower retention

Layer 4: Hierarchical Memory Tiering

Working Memory (registers)  → Always in context
FIFO Queue (L1/L2 cache)    → Recent exchanges
Archival Memory (disk)      → Semantic search only

Implementation Workflow

Check semantic cache before any LLM call
Compress context using appropriate level
Structure placement - critical info at boundaries
Tier management - evict low-importance content
Cache response for future queries

Compression Decision Matrix

| Context Size | Latency Need | Accuracy Need | Strategy | |--------------|--------------|---------------|----------| | <10K tokens | Any | Any | No compression | | 10K-50K | Low | High | Light (2-3x) | | 10K-50K | High | Medium | Moderate (5-7x) | | 50K-100K | Any | Medium | Aggressive (10-20x) | | >100K | Any | Any | Hierarchical + Aggressive |

Key Patterns

Attention Sink Preservation

For streaming/long sessions, preserve first 4 tokens as attention sinks:

[attention_sinks (4 tokens)] + [rolling_window (window - 4)]

This maintains model coherence over infinite context.

Hybrid Search for RAG

Hybrid = Dense (semantic) + Sparse (BM25)
Fusion = Reciprocal Rank Fusion (RRF)

Achieves 50-100x document reduction with maintained relevance.

Metrics to Track

Cache hit rate (target: >60%)
Compression ratio achieved
Accuracy impact (sample validation)
Token savings per session
Latency impact

Integration Points

Pre-prompt: Apply compression
Post-response: Cache result
Session start: Load cached context
Memory pressure: Tier eviction

Based on LLMLingua, GPTCache, MemGPT, and StreamingLLM research

Want to see if miosa-osa/prompt-cache-optimizer would activate on your phrasing? Try it before you install — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/27/2026, 12:38 UTC · 22 cases

Lift

+36%

82% vs 45%

Agent turns

0%

1.0 vs 1.0

Tokens

-6%

2,259.41 vs 2,411.27

fail → pass · 8pass → pass · 10fail → fail · 4

Per-case results (22)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-05 Our deployment team wants to cut memory usage when hosting a 70B parameter model on limited GPU VRAM. What quantization and parameter-efficient tuning techniques should we apply?	pass→pass	18,242	19,968	+9%	1	1	0%	3,174	4,135	+30%	0	0	—
case-06 How should we structure step-by-step reasoning instructions and few-shot examples within our prompt templates to maximize mathematical problem-solving performance?	fail→fail	21,524	20,383	-5%	1	1	0%	3,640	4,111	+13%	0	0	—
case-15 What primary operation must be executed first before running prompt compression or making an LLM API call?	fail→pass	9,610	2,617	-73%	1	1	0%	1,415	1,100	-22%	0	0	—
case-01 I'm running an LLM feature processing around 35K tokens per request. We need fast responses and cannot afford a drop in output accuracy. Can you provide a step-by-step optimization plan for managing this prompt context, including caching and compression recommendations tailored to these operational constraints?	fail→fail	19,970	21,586	+8%	1	1	0%	3,192	3,748	+17%	0	0	—
case-02 We are building a multi-turn conversational system where session memory frequently scales beyond 120k tokens. Could you outline a strategy to reduce our token expenses using context tiering, prompt compression, and strategic positioning of key information?	fail→pass	25,119	30,096	+20%	1	1	0%	3,832	5,061	+32%	0	0	—
case-03 Our enterprise application receives many repeated queries against large reference manuals. Please lay out a pre-execution workflow for our prompt pipeline that checks for cached answers and slims down retrieved document payloads before making the model call.	fail→fail	18,174	23,801	+31%	1	1	0%	3,020	4,185	+39%	0	0	—
case-04 We are developing a custom biomedical text parser. How should we build a Byte-Pair Encoding subword vocabulary for optimal tokenization of clinical terms?	fail→fail	24,312	24,863	+2%	1	1	0%	3,379	4,106	+22%	0	0	—
case-07 We process payloads of 30K tokens where response latency must be minimal, but output fidelity can accept minor trade-offs. Which compression ratio level should our pipeline select?	fail→pass	13,618	6,771	-50%	1	1	0%	2,313	1,659	-28%	0	0	—
case-08 Our application sends brief requests containing under 8K tokens. What compression reduction strategy should be applied to these prompts?	fail→pass	15,331	5,728	-63%	1	1	0%	2,264	1,575	-30%	0	0	—
case-09 We have context payloads measuring 75K tokens with medium accuracy requirements. What compression profile is specified for this size bracket?	fail→pass	11,322	4,005	-65%	1	1	0%	1,579	1,292	-18%	0	0	—
case-10 When maintaining long streaming sessions with a rolling window context, how many initial tokens must be preserved at the prefix to maintain attention stability?	pass→pass	14,193	3,727	-74%	1	1	0%	2,184	1,242	-43%	0	0	—
case-11 We are blending dense vector similarity scores with sparse BM25 lexical scores for RAG document retrieval. Which ranking algorithm should synthesize these lists?	pass→pass	16,277	13,299	-18%	1	1	0%	2,632	3,076	+17%	0	0	—
case-12 To prevent the model from missing essential background data in a 60K prompt context, where should the vital information be placed?	pass→pass	10,481	8,061	-23%	1	1	0%	1,535	1,936	+26%	0	0	—
case-13 In a multi-tier context memory architecture, where should active session registers and system instructions be stored to guarantee they remain in context?	pass→pass	16,008	4,919	-69%	1	1	0%	2,527	1,591	-37%	0	0	—
case-14 Where should historical user conversation turns be persisted when they exceed context bounds and are accessed solely through similarity search?	pass→pass	15,432	7,897	-49%	1	1	0%	2,440	1,757	-28%	0	0	—
case-16 When setting up operational monitoring metrics for a context caching layer, what objective target percentage should be set for cache hits?	pass→pass	16,689	8,344	-50%	1	1	0%	2,077	2,116	+2%	0	0	—
case-17 What is the expected maximum accuracy degradation when applying light prompt context compression to incoming prompts?	fail→pass	13,906	2,346	-83%	1	1	0%	2,126	1,035	-51%	0	0	—
case-18 How much lower is model retention when critical context items are placed in the middle of long context windows compared to the boundaries?	pass→pass	13,478	6,367	-53%	1	1	0%	2,133	1,594	-25%	0	0	—
case-19 What magnitude of document payload reduction is achievable when implementing hybrid fusion search for RAG retrieval?	fail→pass	14,720	2,245	-85%	1	1	0%	2,283	1,059	-54%	0	0	—
case-20 Which neural prompt compression framework should be used to achieve small 2x to 3x token reductions with minimal accuracy impact?	pass→pass	16,442	4,212	-74%	1	1	0%	2,735	1,277	-53%	0	0	—
case-21 What percentage of token expenditure is saved when an incoming query results in a direct semantic cache hit?	pass→pass	20,520	2,566	-87%	1	1	0%	1,657	977	-41%	0	0	—
case-22 In a multi-level context memory system, what tier holds recent message exchanges right after working memory?	fail→pass	5,890	2,454	-58%	1	1	0%	911	1,075	+18%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +36 percentage points is the difference between those two pass rates over the 22 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 45.5%
Pass rate with the skill: 81.8%
Cases attempted: 22
Cases counted toward lift: 22
Cases that regressed: 0
Skill version measured: v1
Run completed: 8/27/2026, 12:38 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-15

Prompt

What primary operation must be executed first before running prompt compression or making an LLM API call?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signalsno signals yet

BenchmarkRun a verified benchmark with positive lift

Live evalNeeds ≥20 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

No signals yet — run a benchmark or collect evaluated traces to earn a score.

Verified on gemini-3.6-flash · re-tested yesterday

Other measured skills in the registry, with their headline benchmark lift.