Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →graniet/vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching.
Passed 20 of 22 test cases with the skill, and 14 of 22 without it.
and 7 more in the full report
6 fixed14 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
Write a minimal Python script to perform offline generation on 'Explain quantum computing' using the meta-llama/Llama-3-8B-Instruct model with a sampling temperature of 0.7 and 256 max tokens.
| Checked | Without skill | With skill |
|---|---|---|
| Imports LLM and SamplingParams from the vllm Python package | Missed | Passed |
| Initializes SamplingParams with temperature 0.7 and max_tokens 256 | Missed | Passed |
| Calls generate method on the LLM instance to produce text output | Missed | Passed |
The instructions your model reads when the skill loads.
This skill is repo-local and stays inactive until explicitly activated.
When the original instructions refer to legacy tool names, use these Kheish mappings:
terminal => bashweb_extract => web_fetch, plus web_search when discovery is neededsearch_files => grep_search and glob_searchbrowser_* tools require a browser-capable surfaced tool or MCP; if none is available, use the closest available surface and say so explicitlyWhen the instructions mention local helper files, resolve them from ${KHEISH_SKILL_DIR}.
vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill/decode requests).
Installation:
pip install vllm
Basic offline inference:
from vllm import LLM, SamplingParams
llm = LLM(model="meta-llama/Llama-3-8B-Instruct")
sampling = SamplingParams(temperature=0.7, max_tokens=256)
outputs = llm.generate(["Explain quantum computing"], sampling)
print(outputs[0].outputs[0].text)
OpenAI-compatible server:
vllm serve meta-llama/Llama-3-8B-Instruct
# Query with OpenAI SDK
test cases passed, vs 14 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.