Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →waybarrios/hf-mem
Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub.
Passed 21 of 21 test cases with the skill, and 6 of 21 without it.
and 27 more in the full report
16 fixed6 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 15 test cases the skill fixed. Same model, same prompt, with and without the skill.
Could you calculate the inference memory requirements for `unsloth/Qwen3.5-397B-A17B-GGUF` specifically targeting the `Q4_K_M` file? I'd like the estimation to include the KV cache for a context window of 8192 and batch size 2, formatted as JSON.
| Checked | Without skill | With skill |
|---|---|---|
| Includes --model-id unsloth/Qwen3.5-397B-A17B-GGUF in the command | Missed | Passed |
| Includes --gguf-file Q4_K_M in the command | Missed | Passed |
| Includes --experimental | Missed | Passed |
| Includes --max-model-len 8192 | Missed | Passed |
| Includes --batch-size 2 | Missed | Passed |
The instructions your model reads when the skill loads.
hf_mem estimates the required memory for inference, including model weights and an optional KV cache, for Safetensors and GGUF for models on the Hugging Face Hub using HTTP Range requests i.e., without downloading or loading any weights locally.
uv installed (for uvx)HF_TOKEN env var or --hf-token flag (for gated or private models only)Run with --model-id pointing to the Hugging Face Hub repository which will check that it either contains Safetensors (via model.safetensors, model.safetensors.index.json if sharded, or model_index.json for Diffusers) or GGUF model weights within.
uvx hf-mem --model-id <model-id> --json-output
If the repository contains GGUF model weights in multiple precisions / quantizations, the estimations will be on a per-file basis, whereas for inference you won't load all of those but rather only a single precision. This being said, for GGUF you might as well need to provide --gguf-file to target the specific file (or path if sharded) you want to run.
uvx hf-mem --model-id <model-id> --gguf-file <file-or-path> --json-output
Additionally, hf-mem comes with an --experimental flag that will also calculate the KV cache memory requirements too, useful for large-language models, meaning it applies to LLMs (...ForCausalLM), VLMs (...ForConditionalGeneration), and GGUF models.
As per the context window, it will be read from the default or overridden with --max-model-len a la vLLM. And, same goes for the KV cache precision, which will default to the model precision unless manually set via --kv-cache-dtype a la vLLM too.
For Safetensors use as:
uvx hf-mem --model-id <model-id> --experimental [--max-model-len N] [--batch-size N] [--kv-cache-dtype auto|bfloat16|fp8|fp8_ds_mla|fp8_e4m3|fp8_e5m2|fp8_inc] --json-output
And, for GGUF use as:
test cases passed, vs 6 / 21 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.