Data & Model Evaluation (agsy-data-and-model-evaluation)

Data & Model Evaluation (agsy-data-and-model-evaluation)

A model is only as credible as its evaluation. AgSy reviewers are systems-modelling experts: they want to see the model tested against independent data, its sensitivity and uncertainty characterized, and the trade-offs the system exhibits — not a single tuned run presented as truth. Model description and choice live in agsy-systems-framing-and-modeling; this skill covers testing and reporting.

When to trigger

• Reporting how well the model reproduces observations
• Running sensitivity / uncertainty analysis
• Building scenario comparisons and trade-off analyses
• A reviewer asked for validation, sensitivity, uncertainty, or alternative scenarios

Evaluation norms AgSy expects

1. Independent evaluation. Compare observed vs. simulated on data not used for calibration. Report standard fit statistics — RMSE, RRMSE, bias/ME, modelling efficiency (NSE), index of agreement (d), R² — and show the 1:1 plot. State what "good enough" means for the decision.
2. Sensitivity analysis. Identify the parameters/inputs that drive outputs (local one-at-a-time and, where feasible, global methods — Morris, Sobol). Report which assumptions matter most.
3. Uncertainty. Propagate uncertainty from inputs, parameters, and structure to the outputs that carry the conclusions. Present ranges/intervals, not point estimates dressed as certainty.
4. Trade-off & scenario analysis. This is where AgSy papers earn their place: compare scenarios or management/design options on multiple objectives (yield, profit, environment, risk) and show the trade-offs and synergies — Pareto fronts, trade-off curves, multi-indicator profiles.
5. Scaling. When you aggregate from field to farm to region, state how, and check that aggregation does not hide compensating errors.

Stochastic & data-driven components

• For Monte Carlo, ABM, or stochastic weather/price generators: set and report seeds, run enough replicates, and report the distribution, not one realization.
• For surrogate/emulator or ML components: validate against the full model and report where they fail.

Reproducibility while you work (not at the end)

• One master workflow regenerates every figure/table from inputs + model runs.
• Pin model version, parameter sets, and software versions; record calibration vs. evaluation splits.
• Keep table/figure numbers matched to script/model outputs — the data/code/model are deposited (see agsy-reproducibility-and-data-policy).