Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →brycewang-stanford/ml-experiment-tracker
Plan reproducible ML experiment runs with parameters and metrics tracking
Passed 12 of 22 test cases with the skill, and 10 of 22 without it.
and 2 more in the full report
4 fixed9 passed either way9 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
Write a Python function to log final evaluation metrics and model artifacts at the end of a training run in MLflow. Metrics should be distinct from intermediate epoch logs.
| Checked | Without skill | With skill |
|---|---|---|
| Prepends final_ to metric key names when calling log_metric | Passed | Passed |
| Calls log_artifact with model path when model_path argument is provided | Missed | Passed |
The instructions your model reads when the skill loads.
A skill for planning, executing, and tracking machine learning experiments with full reproducibility. Covers experiment design, hyperparameter management, metric logging, model versioning, and comparison across runs to support rigorous ML research.
Machine learning research involves running dozens or hundreds of experiments with varying architectures, hyperparameters, data splits, and preprocessing pipelines. Without systematic tracking, it becomes impossible to reproduce results, compare configurations, or identify which changes actually improved performance. This skill provides a structured methodology for experiment management that aligns with academic standards for reproducible ML research.
The approach is framework-agnostic but demonstrates integration with MLflow, Weights & Biases, and plain file-based logging. It emphasizes the practices needed for publications: complete hyperparameter documentation, statistical significance testing across runs, and artifact management for model checkpoints and evaluation outputs.
Before writing any training code, document the experiment plan:
# experiment_plan.yaml
experiment:
name: "transformer-sentiment-analysis-v3"
hypothesis: "Adding relative positional encoding improves F1 on long reviews (>512 tokens)"
dataset:
name: "imdb-extended"
version: "2025.1"
splits: {train: 0.8, val: 0.1, test: 0.1}
stratify_by: "label"
random_seed: 42
baselines:
- name: "bert-base-uncased"
checkpoint: "bert-base-uncased"
- name: "roberta-base"
checkpoint: "roberta-base"
variables:
independent:
- positional_encoding: ["absolute", "relative", "rotary"]
controlled:
- learning_rate: 2e-5
- batch_size: 32
test cases passed, vs 10 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.