Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Early Stopping Callback · dicklesworthstone | DecimalAI

dicklesworthstone/early-stopping-callback

SkillScore91Maintained

Manage early stopping callback operations. Auto-activating skill for ML Training. Triggers on: early stopping callback, early stopping callback Part of the ML Training skill category. Use when working with early stopping callback functionality. Trigger with phrases like "early stopping callback", "early callback", "early".

by dicklesworthstone

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

AI Engineer pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/dicklesworthstone-early-stopping-callback/SKILL.md

decimalai skills pull dicklesworthstone-early-stopping-callback --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/dicklesworthstone-early-stopping-callback/SKILL.md https://app.decimal.ai/s/dicklesworthstone-early-stopping-callback/SKILL.md

Raw file: /s/dicklesworthstone-early-stopping-callback/SKILL.md ↗ · machine-readable summary at /s/dicklesworthstone-early-stopping-callback.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

22 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 25d ago

Yes — +5% lift · at higher cost+32% tokens0% turns91% of whole cases pass with skill (20/22)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-01	✗→✓	▲ Improved	34%	0%
case-04	✓→✓	＝ Same ✓	30%	0%
case-05	✓→✓	＝ Same ✓	16%	0%
case-06	✓→✓	＝ Same ✓	43%	0%
case-07	✓→✓	＝ Same ✓	38%	0%

Showing 5 of 22 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/10/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

dicklesworthstone / pi_agent_rust

tests/ext_conformance/artifacts/plugins-community/skills/07-ml-training/early-stopping-callback/SKILL.md

View on GitHub ↗

License MITImported 8/10/2026Version v1

Skill contents

SKILL.md

Early Stopping Callback

Overview

This skill provides automated assistance for early stopping callback tasks within the ML Training domain.

When to Use

This skill activates automatically when you:

Mention "early stopping callback" in your request
Ask about early stopping callback patterns or best practices
Need help with machine learning training skills covering data preparation, model training, hyperparameter tuning, and experiment tracking.

Instructions

Provides step-by-step guidance for early stopping callback
Follows industry best practices and patterns
Generates production-ready code and configurations
Validates outputs against common standards

Examples

Example: Basic Usage Request: "Help me with early stopping callback" Result: Provides step-by-step guidance and generates appropriate configurations

Prerequisites

Relevant development environment configured
Access to necessary tools and services
Basic understanding of ml training concepts

Output

Generated configurations and code
Best practice recommendations
Validation results

Error Handling

| Error | Cause | Solution | |-------|-------|----------| | Configuration invalid | Missing required fields | Check documentation for required parameters | | Tool not found | Dependency not installed | Install required tools per prerequisites | | Permission denied | Insufficient access | Verify credentials and permissions |

Resources

Official documentation for related tools
Best practices guides
Community examples and tutorials

Related Skills

Part of the ML Training skill category. Tags: ml, training, pytorch, tensorflow, sklearn

Illustrative Want to see what dicklesworthstone/early-stopping-callback would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/29/2026, 18:46 UTC · 22 cases

Lift

+5%

91% vs 86%

Agent turns

1.0 vs 1.0

Tokens

+32%

1,981.14 vs 1,504.86

fail → pass · 1pass → pass · 19fail → fail · 2

Per-case results (22)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 I am setting up a tf.keras training pipeline for an image classification model. A colleague recommended configuring `tf.keras.callbacks.EarlyStopping` with `monitor='loss'` to stop training whenever the loss curve flattens. Write the Python code for this callback configuration.	fail→pass	9,339	22,020	+136%	1	1	0%	1,650	2,213	+34%	0	0	—
case-02 When configuring `tf.keras.callbacks.EarlyStopping` for a neural network with `patience=5`, a tutorial suggests omitting `restore_best_weights` so that the model in memory retains the final epoch weights after training stops. Provide the Python code to instantiate the callback correctly.	fail→fail	3,859	6,325	+64%	1	1	0%	716	1,527	+113%	0	0	—
case-03 I am implementing a custom PyTorch EarlyStopping callback class. I want to trigger stopping immediately on the very first epoch where validation loss increases, without tracking consecutive non-improving epochs or enforcing a minimum change threshold. Write the custom PyTorch EarlyStopping class implementation.	fail→fail	10,056	10,292	+2%	1	1	0%	2,121	2,516	+19%	0	0	—
case-04 In PyTorch Lightning, I am creating an `EarlyStopping` callback monitoring `val_accuracy`. A teammate recommended setting `mode='min'` because loss callbacks use min. Write the Python code to instantiate this callback.	pass→pass	26,505	6,364	-76%	1	1	0%	1,174	1,526	+30%	0	0	—
case-05 I am training an XGBoost classifier using `xgb.train` with `early_stopping_rounds=10`. After training completes, a code snippet uses `bst.predict(dtest)` without passing the best iteration, assuming XGBoost defaults to the best tree model. Write the correct prediction code.	pass→pass	8,259	9,670	+17%	1	1	0%	1,578	1,827	+16%	0	0	—
case-06 I am calling `lgb.train` with `callbacks=[lgb.early_stopping(stopping_rounds=15)]`. A code snippet omits the `valid_sets` parameter, assuming LightGBM will monitor training metrics. Write the complete `lgb.train` call.	pass→pass	8,557	8,031	-6%	1	1	0%	1,357	1,943	+43%	0	0	—
case-07 I am adding `EarlyStoppingCallback` to a Hugging Face `Trainer` instance. The default `TrainingArguments` has `eval_strategy='no'`. Write the `TrainingArguments` configuration so early stopping can measure model performance.	pass→pass	9,275	11,389	+23%	1	1	0%	1,868	2,569	+38%	0	0	—
case-08 I am training `SGDClassifier` in scikit-learn. To speed up fitting, a snippet sets `early_stopping=True` but leaves `validation_fraction=0.0` and `n_iter_no_change=1`. Write the corrected `SGDClassifier` initialization code.	pass→pass	6,634	9,436	+42%	1	1	0%	1,205	2,066	+71%	0	0	—
case-09 I am writing a custom PyTorch early stopping tracker. When a lower validation loss is encountered, my method runs `self.best_weights = model.state_dict()`. Write the corrected assignment so subsequent in-place model updates do not mutate `best_weights`.	pass→pass	8,053	6,208	-23%	1	1	0%	1,371	1,362	-1%	0	0	—
case-10 In tf.keras, I want training to halt if validation accuracy fails to reach an absolute benchmark of 0.85 within 10 epochs. A suggestion was made to set `min_delta=0.85`. Write the correct `tf.keras.callbacks.EarlyStopping` instantiation.	pass→pass	12,855	7,721	-40%	1	1	0%	2,383	1,694	-29%	0	0	—
case-11 I am setting up `EarlyStopping(monitor='val/loss')` in PyTorch Lightning. However, in my `LightningModule.validation_step`, I log metrics using `self.log('validation_loss', loss)`. Write the corrected `EarlyStopping` initialization or logging call so the metric keys match.	pass→pass	5,540	6,845	+24%	1	1	0%	864	1,633	+89%	0	0	—
case-12 I am training a `CatBoostClassifier` on tabular data. A code snippet sets `early_stopping_rounds=20` in `fit()` but passes only `X_train` and `y_train`. Write the correct `fit()` method call.	pass→pass	6,768	6,150	-9%	1	1	0%	1,081	1,420	+31%	0	0	—
case-13 When building a custom PyTorch early stopping callback, I need to prevent infinite epoch loops when gradients explode and validation loss becomes `NaN`. Write the check inside the callback step method.	pass→pass	9,810	15,408	+57%	1	1	0%	1,849	3,602	+95%	0	0	—
case-14 In Keras, when fine-tuning a pretrained Convolutional Neural Network, severe initial validation loss spikes in the first 5 epochs trigger `EarlyStopping` prematurely. Instead of increasing global patience, write the `tf.keras.callbacks.EarlyStopping` setup that postpones callback monitoring until epoch 5.	pass→pass	7,778	9,433	+21%	1	1	0%	1,515	2,107	+39%	0	0	—
case-15 I am using `MLPClassifier` from scikit-learn for multi-class tabular classification. I want the classifier to automatically split training data and stop training when validation score fails to improve. Write the `MLPClassifier` instantiation.	pass→pass	10,190	4,820	-53%	1	1	0%	1,364	1,286	-6%	0	0	—
case-16 I instantiated a custom `EarlyStopping` object named `stopper` inside a PyTorch training script. Inside the epoch loop after running validation, I call `stopper(val_loss, model)`. Write the outer loop code to stop training when `stopper.early_stop` is set.	pass→pass	6,858	6,123	-11%	1	1	0%	1,164	1,487	+28%	0	0	—
case-17 When configuring `TrainingArguments` with `EarlyStoppingCallback` for a text classification task in Hugging Face Transformers, I specify `metric_for_best_model='f1'` and `load_best_model_at_end=True`. A draft script sets `greater_is_better=False`. Provide the corrected `TrainingArguments` setup.	pass→pass	7,639	9,046	+18%	1	1	0%	1,532	2,111	+38%	0	0	—
case-18 When using `lgb.early_stopping` in LightGBM during hyperparameter tuning, I want to disable early stopping console output for every iteration. Write the `lgb.early_stopping` callback initialization.	pass→pass	6,027	6,022	-0%	1	1	0%	1,034	1,449	+40%	0	0	—
case-19 In PyTorch Lightning, I want `ModelCheckpoint` to save the model checkpoint corresponding to the optimal epoch identified by `EarlyStopping(monitor='val_loss', mode='min')`. Write the `ModelCheckpoint` instantiation.	pass→pass	6,678	5,618	-16%	1	1	0%	1,068	1,412	+32%	0	0	—
case-20 I am building a PyTorch training pipeline for a ResNet model. I need to adjust the learning rate dynamically following a cosine curve across 100 epochs using `torch.optim.lr_scheduler.CosineAnnealingLR`. Write the code to configure the optimizer and scheduler.	pass→pass	8,155	9,389	+15%	1	1	0%	1,617	2,321	+44%	0	0	—
case-21 I have an imbalanced classification dataset in pandas and scikit-learn. I need to generate 5 train and validation folds while preserving class proportions using `StratifiedKFold`. Write the fold splitting code.	pass→pass	8,625	10,275	+19%	1	1	0%	1,675	2,512	+50%	0	0	—
case-22 I am training a LightGBM model and want to log hyperparameters and evaluation metrics to an MLflow tracking server using `mlflow.log_params` and `mlflow.log_metrics`. Write the Python code managing the MLflow run.	pass→pass	13,523	15,829	+17%	1	1	0%	2,921	3,002	+3%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +5 percentage points is the difference between those two pass rates over the 22 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 86.4%
Pass rate with the skill: 90.9%
Cases attempted: 22
Cases counted toward lift: 22
Cases that regressed: 0
Skill version measured: v1
Run completed: 8/29/2026, 18:46 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-01

Prompt

I am setting up a tf.keras training pipeline for an image classification model. A colleague recommended configuring `tf.keras.callbacks.EarlyStopping` with `monitor='loss'` to stop training whenever the loss curve flattens. Write the Python code for this callback configuration.

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run91

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+30% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 24 days ago

Other measured skills in the registry, with their headline benchmark lift.