Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →graniet/lm-evaluation-harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).
Passed 21 of 22 test cases with the skill, and 17 of 22 without it.
and 1 more in the full report
4 fixed17 passed either way1 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
Before running HumanEval code execution benchmarking, what extra pip package must be installed locally to provide the coding dataset environment?
| Checked | Without skill | With skill |
|---|---|---|
| The instruction specifies running 'pip install human-eval' | Missed | Passed |
The instructions your model reads when the skill loads.
This skill is repo-local and stays inactive until explicitly activated.
When the original instructions refer to legacy tool names, use these Kheish mappings:
terminal => bashweb_extract => web_fetch, plus web_search when discovery is neededsearch_files => grep_search and glob_searchbrowser_* tools require a browser-capable surfaced tool or MCP; if none is available, use the closest available surface and say so explicitlyWhen the instructions mention local helper files, resolve them from ${KHEISH_SKILL_DIR}.
lm-evaluation-harness evaluates LLMs across 60+ academic benchmarks using standardized prompts and metrics.
Installation:
pip install lm-eval
Evaluate any HuggingFace model:
lm_eval --model hf \
--model_args pretrained=meta-llama/Llama-2-7b-hf \
--tasks mmlu,gsm8k,hellaswag \
--device cuda:0 \
--batch_size 8
View available tasks:
lm_eval --tasks list
test cases passed, vs 17 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.