Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.
SkillScore91Maintained
Automatic
◆AI Engineer pack
Last verified 8/20/2026
What it changes
Passed 20 of 22 test cases with the skill, and 14 of 22 without it.
Without skill64%
With skill91%
Checks it passed that the model alone missed
Uses hf auth login for CLI authentication
Includes apps=llama.cpp in the query string
Includes apps=llama.cpp in the search query
and 5 more in the full report
6 fixed14 passed either way2 still failing0 broke
Pass-rate lift
+27 pp
Tokens
+24%
Agent turns
0%
Response time
-40%
Model gemini-3.6-flashMeasured 8/20/2026v120 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 6
Prompt
When linking directly to a Hugging Face model repository page to view recommended quants and launcher snippets for llama.cpp, what query parameter should be appended to the URL?
What was checked, without and with the skill
Checked
Without skill
With skill
Appends ?local-app=llama.cpp to the model page URL
Missed
Passed
case-18 · gemini-3.6-flash · 8/20/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md108 lines4 files
Hugging Face Local Models
Search the Hugging Face Hub for llama.cpp-compatible GGUF repos, choose the right quant, and launch the model with llama-cli or llama-server.
Default Workflow
Search the Hub with apps=llama.cpp.
Open https://huggingface.co/<repo>?local-app=llama.cpp.
Prefer the exact HF local-app snippet and quant recommendation when it is visible.
Confirm exact .gguf filenames with https://huggingface.co/api/models/<repo>/tree/main?recursive=true.
Launch with llama-cli -hf <repo>:<QUANT> or llama-server -hf <repo>:<QUANT>.
Fall back to --hf-repo plus --hf-file when the repo uses custom file naming.
Convert from Transformers weights only if the repo does not already expose GGUF files.
Quick Start
Install llama.cpp
brew install llama.cpp
winget install llama.cpp
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
make