fix(hype): score the calibration objective on the calibration period

[DarriEy]

Problem

HYPEWorker.calculate_metrics restricted scoring to the calibration period like this:

calib_period = None
try:
    calib_period = config.optimization.calibration_period
except (AttributeError, TypeError):
    pass
if calib_period:
    ...   # filter sim/obs to the window

calibration_period lives on the domain section (CALIBRATION_PERIOD) — OptimizationConfig has no such field. So the AttributeError fired on every call, was caught and passed over, calib_period stayed None, the filter was skipped, and nothing was logged.

Every HYPE calibration therefore scored the entire overlapping record, evaluation period included.

Two consequences — the same pair recently fixed across the JAX model packages, reached by a different route:

1. Evaluation-period leakage. The optimizer trained on held-out data, so the split-sample test was not honest.
2. Mismatched reporting. The score the optimizer reported covered a different window than the Calib_* metrics written at final evaluation.

On the P3 Bow-at-Banff calibration ensemble, all 14 HYPE runs disagreed with their own final evaluation, by 3e-3 to 5.9e-2 — across every algorithm, not just the gradient-based ones. That breadth is what distinguishes this from the gradient-path bugs.

Fix

Resolve through the canonical CALIBRATION_PERIOD key, exactly as _calculate_multi_gauge_metrics in this same class already does (line 637), with a typed-config fallback.

When no period resolves, warn instead of silently scoring everything. The silence is what hid this for so long: a try/except AttributeError: pass around a config read gives identical behaviour whether the key is absent, misspelled, or moved.

Verification

HYPE/abc on Bow-at-Banff, before and after:

	best_score	Calib_KGE	|diff|	Eval_KGE
before	0.824691	0.819599	5.1e-03	0.805
after	0.832525	0.832523	2.0e-06	0.550

The residual now matches the other models (HBV 5.4e-07, TOPMODEL 3.6e-09, FUSE 9.6e-07).

The Eval_KGE drop from 0.805 to 0.550 is the point, not a regression. The evaluation period was previously part of what the optimizer trained on, so held-out performance was overstated. Any previously published HYPE evaluation-period number is affected.

This is 1 of 15 HYPE configs; the remaining 14 are re-running and I'll follow up if any disagrees.

Related

Companion to the calibration-window work in #392 and the seven model-package PRs (jHBV, jTOPMODEL, jSAC-SMA, jXAJ, jHEC-HMS, jFUSE, cFUSE), all merged.

🤖 Generated with Claude Code