Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mkurman/vllm
Fast LLM inference engine. PagedAttention, continuous batching, tensor parallelism, speculative decoding, and prefix caching. OpenAI-compatible API server. Supports Llama, Mistral, Qwen, DeepSeek, and hundreds of models.
Passed 22 of 23 test cases with the skill, and 21 of 23 without it.
2 fixed20 passed either way0 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 2 test cases the skill fixed. Same model, same prompt, with and without the skill.
Given an initialized engine instance llm and sampling configuration params, write Python code to generate text for prompt 'What is the capital of France?' without manually tokenizing inputs.
| Checked | Without skill | With skill |
|---|---|---|
| Calls llm.generate with prompt list | Missed | Passed |
| Passes params to generate | Passed | Passed |
The instructions your model reads when the skill loads.
vLLM is a high-throughput, memory-efficient LLM inference engine featuring PagedAttention (near-zero memory waste), continuous batching, tensor parallelism, speculative decoding, prefix caching, and an OpenAI-compatible API.
uv pip install vllm
from vllm import LLM, SamplingParams
llm = LLM(model="Qwen/Qwen2.5-1.5B-Instruct")
params = SamplingParams(temperature=0.7, top_p=0.9, max_tokens=512)
outputs = llm.generate(["What is the capital of France?"], params)
for o in outputs:
print(o.outputs[0].text)
vllm serve Qwen/Qwen2.5-1.5B-Instruct --port 8000
# OpenAI client:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "Qwen/Qwen2.5-1.5B-Instruct", "messages": [{"role": "user", "content": "Hello!"}]}'
llm = LLM(model="meta-llama/Llama-3.1-8B", tensor_parallel_size=2)
test cases passed, vs 21 / 23 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.