For ensemble-based scoring rules, we need flexible handling of missing values in ensemble members. Currently, ensemble-based metrics such as the CRPS return NaN if there is one or more NaNs in the ensemble members. It may be the case that users have an ensemble with a few NaNs (e.g. with lagged ensembles you have NaNs for some timestamps) but still want to get a valid score.
Proposed Solution
We plan on allowing users to specify a nan_policy argument to ensemble metrics that controls how NaN values are handled. The argument will take three possible values:
- propagate (default): current behavior - return NaN if any ensemble member is NaN
- omit: ignores NaN values during ensemble metric computation
- raise: raise an error if NaN values are encountered
For ensemble-based scoring rules, we need flexible handling of missing values in ensemble members. Currently, ensemble-based metrics such as the CRPS return NaN if there is one or more NaNs in the ensemble members. It may be the case that users have an ensemble with a few NaNs (e.g. with lagged ensembles you have NaNs for some timestamps) but still want to get a valid score.
Proposed Solution
We plan on allowing users to specify a
nan_policyargument to ensemble metrics that controls how NaN values are handled. The argument will take three possible values: