Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mkurman/llama-cpp
LLM inference in C/C++ with Python bindings. GPU acceleration via CUDA/Metal/Vulkan, 2-8 bit quantization (GGUF), KV cache, and grammar-based sampling. Run Llama, Mistral, Gemma, Phi locally.
Passed 22 of 22 test cases with the skill, and 21 of 22 without it.
1 fixed21 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
A test case the skill fixed. Same model, same prompt, with and without the skill.
I need the pip command to install llama-cpp-python with CUDA acceleration enabled, followed by a minimal Python example that loads a GGUF file and generates text for a short query.
| Checked | Without skill | With skill |
|---|---|---|
| Uses the extra index URL https://abetlen.github.io/llama-cpp-python/whl/cu124 in the pip install command | Missed | Passed |
| Includes --extra-index-url option rather than build flags like CMAKE_ARGS | Missed | Passed |
The instructions your model reads when the skill loads.
llama.cpp is a C++ inference engine for LLMs optimized for CPU and Apple Silicon. Runs GGUF-format models (Llama, Mistral, Qwen, Gemma, Phi, DeepSeek, etc.) with quantization from Q2 to Q8. Supports GPU offloading, batch inference, and OpenAI-compatible server.
uv pip install llama-cpp-python
# For CUDA: pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124
from llama_cpp import Llama
llm = Llama(model_path="qwen2.5-1.5b-instruct-q4_k_m.gguf")
output = llm("Q: What is machine learning?
A:", max_tokens=128)
print(output["choices"][0]["text"])
llm = Llama(model_path="model.gguf", chat_format="chatml")
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain gradient descent."},
]
)
print(response["choices"][0]["message"]["content"])
llm = Llama(
model_path="model.gguf",
test cases passed, vs 21 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.