hydra-zen

Overview

hydra-zen is a Python-first layer on top of Hydra that removes most Hydra-specific boilerplate and eliminates hand-written YAML configs. It dynamically generates structured dataclass configs from functions, classes, and call signatures, then integrates them into Hydra workflows. Use this skill when you want typed, composable, reproducible experiment configuration without maintaining large YAML trees.

When to Use This Skill

This skill should be used when:

• You want Hydra benefits without hand-writing YAML configs
• You need type-safe, dataclass-driven experiment configuration
• You want to generate configs directly from Python call signatures
• You want to reduce Hydra boilerplate in ML/research codebases
• You need Pythonic launch/store APIs for configurable workflows
• You are building reproducible training scripts with complex parameterization

Core Capabilities

1. Installation

bash
pip install hydra-zen

2. builds() — Generate Configs from Callables

python
from hydra_zen import builds
from torch.optim import AdamW

AdamWConf = builds(AdamW, lr=1e-3, weight_decay=1e-2)

# AdamWConf is a dataclass config that Hydra can instantiate

You can create configs for:

• classes
• functions
• callables
• nested object graphs

3. instantiate() — Materialize from Config

python
from hydra_zen import builds, instantiate
from torch.optim import AdamW

AdamWConf = builds(AdamW, lr=1e-3, weight_decay=1e-2)
optimizer = instantiate(AdamWConf)

4. make_config() — Typed Ad Hoc Configs

python
from hydra_zen import make_config

TrainConfig = make_config(
    learning_rate=1e-3,
    batch_size=64,
    max_epochs=20,
    model_name="resnet50",
)

cfg = TrainConfig()
print(cfg.learning_rate)

Useful when you just need a typed config object without defining a full dataclass manually.

5. store() — Register Configs with Hydra

python
from hydra_zen import store, builds
from torch.optim import AdamW, SGD

store(group="optimizer")(
    builds(AdamW, lr=1e-3),
    name="adamw"
)

store(group="optimizer")(
    builds(SGD, lr=0.1, momentum=0.9),
    name="sgd"
)

This gives you Hydra config-group behavior without maintaining YAML files.

6. zen() — Wrap Task Functions

python
from hydra_zen import zen

def train(model, optimizer, epochs: int = 10):
    print(model, optimizer, epochs)

train_task = zen(train)
train_task(model="resnet50", optimizer="adamw", epochs=20)

zen() helps bridge normal Python functions and Hydra-configurable execution.

7. End-to-End Example

python
from hydra_zen import builds, store, zen
from torch.optim import AdamW
from torchvision.models import resnet50

ModelConf = builds(resnet50, pretrained=False, num_classes=10)
OptimConf = builds(AdamW, lr=1e-3)

store(group="model", name="resnet50")(ModelConf)
store(group="optimizer", name="adamw")(OptimConf)

@zen
def train(model, optimizer, epochs=10):
    print("Model:", model)
    print("Optimizer:", optimizer)
    print("Epochs:", epochs)

if __name__ == "__main__":
    train.hydra_main(
        config_name=None,
        version_base="1.3",
    )

8. launch() — Programmatic Hydra Runs

python
from hydra_zen import builds, launch

def train(lr: float, batch_size: int):
    return {"lr": lr, "batch_size": batch_size}

Conf = builds(train, lr=1e-3, batch_size=64)
job = launch(Conf)
print(job.return_value)

Useful for notebook workflows, testing, and programmatic sweep orchestration.

9. Nested Config Composition

python
from hydra_zen import builds, instantiate
from torch.optim import AdamW
from torchvision.models import resnet18

ModelConf = builds(resnet18, num_classes=100)
OptimConf = builds(AdamW, lr=1e-4)

ExperimentConf = builds(
    dict,
    model=ModelConf,
    optimizer=OptimConf,
    seed=42,
    hydra_convert="all",
)

exp = instantiate(ExperimentConf)
print(exp["seed"])

10. ML Workflow Pattern

python
from hydra_zen import builds, store, zen
from pytorch_lightning import Trainer

TrainerConf = builds(
    Trainer,
    max_epochs=50,
    accelerator="auto",
    devices=1,
)

store(group="trainer", name="default")(TrainerConf)

@zen
def run_training(trainer, model, datamodule):
    trainer.fit(model, datamodule)

This works especially well for:

• PyTorch Lightning
• optimizer / scheduler registries
• model family registries
• experiment launchers
• notebook-driven experimentation

Key Patterns

1. Prefer builds() over handwritten YAML for Python-heavy projects.
2. Use store() to recreate Hydra config groups with less maintenance.
3. Use zen() to wrap normal task functions into config-driven workflows.
4. Use launch() in tests/notebooks when CLI Hydra feels heavy.
5. hydra-zen shines when your source of truth is Python code, not config files.

References

• hydra-zen documentation
• API reference
• PyTorch Lightning tutorial
• Repository