Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →orchestra-research/evaluating-cosmos-policy
Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments.
Passed 23 of 23 test cases with the skill, and 5 of 23 without it.
and 29 more in the full report
20 fixed5 passed either way0 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 18 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are setting up a virtual environment for RoboCasa benchmarking under CUDA 12.8. Which dependency group flag should be supplied to `uv sync`, and what Python module serves as the official evaluation entrypoint?
| Checked | Without skill | With skill |
|---|---|---|
| The uv synchronization specifies --group robocasa. | Missed | Passed |
| The Python execution module is cosmos_policy.experiments.robot.robocasa.run_robocasa_eval. | Missed | Passed |
The instructions your model reads when the skill loads.
Evaluation workflows for NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments from the public cosmos-policy repository. Covers blank-machine setup, headless GPU evaluation, and inference profiling.
Run a minimal LIBERO evaluation using the official public eval module:
uv run --extra cu128 --group libero --python 3.10 \
python -m cosmos_policy.experiments.robot.libero.run_libero_eval \
--config cosmos_predict2_2b_480p_libero__inference_only \
--ckpt_path nvidia/Cosmos-Policy-LIBERO-Predict2-2B \
--config_file cosmos_policy/config/config.py \
--use_wrist_image True \
--use_proprio True \
--normalize_proprio True \
--unnormalize_actions True \
--dataset_stats_path nvidia/Cosmos-Policy-LIBERO-Predict2-2B/libero_dataset_statistics.json \
--t5_text_embeddings_path nvidia/Cosmos-Policy-LIBERO-Predict2-2B/libero_t5_embeddings.pkl \
--trained_with_image_aug True \
--chunk_size 16 \
--num_open_loop_steps 16 \
--task_suite_name libero_10 \
--num_trials_per_task 1 \
--local_log_dir cosmos_policy/experiments/robot/libero/logs/ \
--seed 195 \
--randomize_seed False \
--deterministic True \
--run_id_note smoke \
--ar_future_prediction False \
--ar_value_prediction False \
--use_jpeg_compression True \
--flip_images True \
--num_denoising_steps_action 5 \
--num_denoising_steps_future_state 1 \
--num_denoising_steps_value 1 \
--data_collection False
test cases passed, vs 5 / 23 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.