Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →google-ai-edge/litert-model-equivalence-test
Validates equivalence between LiteRT models (litert_lm) and PyTorch models (transformers).
Passed 19 of 20 test cases with the skill, and 5 of 20 without it.
and 17 more in the full report
17 fixed5 passed either way0 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 14 test cases the skill fixed. Same model, same prompt, with and without the skill.
I want to run the LiteRT equivalence validation tool for Hugging Face model `google/gemma-3-270m-it`, saving the intermediate export files to `/tmp/model_export_dir` instead of the default location. What is the command?
| Checked | Without skill | With skill |
|---|---|---|
| Uses Bazel target //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test | Missed | Passed |
| Uses --work_dir=/tmp/model_export_dir flag | Missed | Passed |
The instructions your model reads when the skill loads.
This skill provides instructions for running equivalence tests between LiteRT models and their PyTorch source models.
Use the equivalence_test script to compare the outputs of a Hugging Face model
and its exported LiteRT version.
Run the test using bazel run from your workspace:
bazel run \
//third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test \
-- \
--model_id={model_id} \
[--prompt={prompt}] \
[--prompt_file={prompt_file}] \
[--max_new_tokens={max_new_tokens}] \
[--max_num_tokens={max_num_tokens}] \
[--work_dir={work_dir}] \
[--externalize_embedder] \
[--single_token_embedder] \
[--split_cache] \
[--backend={backend}]
--model_id: The Hugging Face model ID to validate (e.g.,
google/gemma-3-270m-it).--prompt: Prompt to test. Specify multiple times for multi-turn
conversations.--prompt_file: Path to a file containing one (complex) prompt. Overrides
--prompt.--max_new_tokens: Maximum new tokens to generate per turn (default: 20).--max_num_tokens: KV cache length for the model (default: 2048).test cases passed, vs 5 / 20 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.