Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →mkurman/optuna
Hyperparameter optimization framework (Optuna). Define-by-run API with automatic search space construction, state-of-the-art samplers (TPE, CMA-ES, NSGA-II, GPSampler), efficient pruning (Median, Hyperband, ASHA), multi-objective optimization, constrained optimization, distributed parallel execution, and visualization dashboard. Integrates with PyTorch, PyTorch Lightning, TensorFlow, Keras, XGBoost, LightGBM, CatBoost, MLflow, W&B, and scikit-learn.
Passed 20 of 22 test cases with the skill, and 18 of 22 without it.
2 fixed18 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 2 test cases the skill fixed. Same model, same prompt, with and without the skill.
I need to run a multi-objective hyperparameter optimization script using Optuna where I evaluate both model accuracy and execution latency simultaneously. I'll be running workers across multiple servers, so I need code that configures the study to connect to a shared PostgreSQL database and retrieve the final Pareto optimal trial configurations.
| Checked | Without skill | With skill |
|---|---|---|
| Configures NSGAIISampler for multi-objective sampling | Missed | Passed |
| Uses postgresql:// storage connection string | Passed | Passed |
| Retrieves Pareto optimal trials using study.best_trials | Passed | Passed |
The instructions your model reads when the skill loads.
---|----------|-------------|
| TPESampler | General ML tuning | Tree-structured Parzen Estimator; default, good for most cases |
| CMAESSampler | Continuous, low-dim (<100) | Covariance Matrix Adaptation; efficient for numeric params |
| NSGAIISampler | Multi-objective (2-3 objectives) | Pareto-front optimization |
| GPSampler | Expensive evaluations | Gaussian Process-based; sample-efficient |
| RandomSampler | Baseline, debugging | Uniform random sampling |
| GridSampler | Small discrete spaces | Exhaustive grid search |
| QMCSampler | Continuous spaces | Quasi-Monte Carlo, better coverage than random |
Usage:
import optuna
sampler = optuna.samplers.TPESampler(seed=42, n_startup_trials=10)
study = optuna.create_study(sampler=sampler)
Stop unpromising trials early to save compute:
def objective(trial):
for epoch in range(100):
accuracy = train_and_evaluate(...)
# Report intermediate value
trial.report(accuracy, epoch)
# Check if should prune
if trial.should_prune():
raise optuna.TrialPruned()
return accuracy
Pruner Selection:
MedianPruner: Prune if trial's intermediate value is below median at same stepHyperbandPruner: Successive halving; efficient for large trial countsSuccessiveHalvingPruner: Similar to Hyperband, simpler configurationThresholdPruner: Prune below absolute thresholdPatientPruner: Prune after N epochs without improvementIntegration with PyTorch Lightning:
test cases passed, vs 18 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.