Integration of MEC workflow #110
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,93 @@ | ||||||
| # yaml-language-server: $schema=../workflow/tools/config.schema.json | ||||||
| description: | | ||||||
| Evaluate skill of Stage E with/without cutoff edges trained with and without subgrid orography. | ||||||
|
|
||||||
| dates: | ||||||
| start: 2025-07-26T00:00 | ||||||
| end: 2025-07-27T00:00 | ||||||
| frequency: 6h | ||||||
|
|
||||||
| runs: | ||||||
| - forecaster: | ||||||
| inference_resources: | ||||||
| slurm_partition: normal-shared | ||||||
| checkpoint: https://service.meteoswiss.ch/mlstore#/experiments/602/runs/c30490b6ba064e4db03b430f3a2595ad | ||||||
| label: stage_E_icon_1km_cutoff_edges_subgrid_horography | ||||||
|
Contributor
There was a problem hiding this comment.
Suggested change
|
||||||
| steps: 0/24/6 | ||||||
| config: resources/inference/configs/sgm-multidataset-forecaster-global-ich1-oper.yaml | ||||||
| extra_requirements: | ||||||
| - git+https://github.com/ecmwf/anemoi-inference.git@e369b1a90313e9701db13f63364a467aa281cf36 | ||||||
|
|
||||||
| baselines: | ||||||
| - baseline: | ||||||
| baseline_id: ICON-CH1-EPS | ||||||
| label: ICON-CH1-ctrl | ||||||
| root: /store_new/mch/msopr/ml/ICON-CH1-EPS | ||||||
| steps: 0/24/6 | ||||||
|
|
||||||
| - baseline: | ||||||
| baseline_id: ICON-CH2-EPS | ||||||
| label: ICON-CH2-ctrl | ||||||
| root: /store_new/mch/msopr/ml/ICON-CH2-EPS | ||||||
| steps: 0/24/6 | ||||||
|
|
||||||
|
|
||||||
| truth: | ||||||
| label: KENDA-CH1 | ||||||
| root: /store_new/mch/msopr/ml/datasets/mch-ich1-1km-2024-2025-1h-pl13-v1.0.zarr | ||||||
|
|
||||||
| experiment: | ||||||
| params: | ||||||
| - T_2M | ||||||
| - TD_2M | ||||||
| - U_10M | ||||||
| - V_10M | ||||||
| - TOT_PREC | ||||||
| stratification: | ||||||
| regions: | ||||||
| - jura | ||||||
| root: /scratch/mch/bhendj/regions/Prognoseregionen_LV95_20220517 | ||||||
| thresholds: | ||||||
| TOT_PREC: | ||||||
| gt: [0.0, 1, 5] | ||||||
| U_10M: | ||||||
| gt: [2.5, 5.0, 10.0] | ||||||
| V_10M: | ||||||
| gt: [2.5, 5.0, 10.0] | ||||||
| T_2M: | ||||||
| lt: [273.15] | ||||||
| gt: [288.15, 298.15] | ||||||
| dashboard: | ||||||
| stratification: | ||||||
| # - init_hour | ||||||
| # - region | ||||||
| - season | ||||||
|
|
||||||
| locations: | ||||||
| output_root: output/ | ||||||
|
|
||||||
| mec: | ||||||
| ekf_root: /store_new/mch/msopr/osm/KENDA-CH1/EKF | ||||||
| mon_synop_root: /scratch/mch/paa/mec/MEC_ML_input/monFiles2025 | ||||||
| ver_synop_root: /scratch/mch/paa/mec/MEC25_I-CH1 | ||||||
|
|
||||||
| ffv2: | ||||||
| experiment_ids: Varda | ||||||
| experiment_description: Varda-Single_suboro | ||||||
| file_description: exp_Varda-Single_2025 | ||||||
| domain_table: /users/paa/01_store/02_FFV2/data/8_CH_box | ||||||
| blacklists: /users/paa/01_store/02_FFV2/data/blacklist | ||||||
|
|
||||||
| profile: | ||||||
| executor: slurm | ||||||
| global_resources: | ||||||
| gpus: 16 | ||||||
| default_resources: | ||||||
| slurm_partition: "postproc" | ||||||
| cpus_per_task: 1 | ||||||
| mem_mb_per_cpu: 1800 | ||||||
| runtime: "1h" | ||||||
| gpus: 0 | ||||||
| jobs: 50 | ||||||
| batch_rules: | ||||||
| plot_forecast_frame: 32 | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
|
|
||
| # variables to verify. | ||
| varnoContinuous 'T2M,TD2M,RH2M,U10M,V10M,PS,FF,DD,GUST_6h,RR_6h' #,RR_6h,N,N_L,N_M,N_H,RAD_GL_1h | ||
|
Contributor
There was a problem hiding this comment. Variables are hardcoded, does this work also with a subset? Should/can this be made configurable?
Author
There was a problem hiding this comment. It works also with a subset. Is there a use case for subsets? We may always want to verify all variables we have.
Contributor
There was a problem hiding this comment. Well, as you say this depends on the observations AND model data. So far we don't have wind gusts for example, so this shouldn't work. If there is internal homogenization (subsettting) of the requested parameters to the intersect of params available from both forecast and ground truth, this won't break with an 'extensive' set, but I still think it would be clearer if specified accordingly.
Author
There was a problem hiding this comment. FFV2 just ignores a variable if it is not there. But I agree it would be clearer to specify only those variables we actually have. |
||
| pecthresholds list('FF'=list('lower'=c(2),'upper'=c(7))) # hit rates (percent correct forecast) for the forecast to hit the observation within the given limits are calculated. | ||
| catthresholds list('T2M'=c(282,292),'FF'=c(2.5,5,10)) # no space allowed between variables | ||
|
Contributor
There was a problem hiding this comment. Any chance we can reuse the thresholds defined in the main config?
Author
There was a problem hiding this comment. Agree, this should go into the config because that will be frequently changed. I will do it.
Contributor
There was a problem hiding this comment. So we could basically specify the most extensive list of parameters availbale from observations and then be done with it? Is this the case now? |
||
|
|
||
| # senseful and short description of the model versions to verify. | ||
| # example: 'D-metno,C-metno_rollout,C-cerra_rollout' | ||
| expIds '{{ experiment_ids }}' | ||
|
|
||
|
|
||
|
|
||
| # location of the feedback files. All feedback files per model must be stored | ||
| # in one directory and the order must correspond to expIds | ||
| # Example: '/src/ffv2/input/,' | ||
| # fdbkDirs '{{ feedback_directories }}' | ||
| # Use a path in the container, to avoid mixups between absolute/relative paths. | ||
| # TODO: Change this once we support having more than one directory. | ||
| fdbkDirs '/src/ffv2/input' | ||
|
|
||
| # should be however many models | ||
| # example: '-1,-1,-1' for 3 models | ||
| veri_ens_member '{{ veri_ens_member }}' | ||
|
|
||
|
|
||
| # the (existing!) output directory (absolute or relative to working directory) | ||
| #outDir '{{ output_directory }}' | ||
| # Use a path in the container, to avoid mixups between absolute/relative paths. | ||
| outDir '/src/ffv2/output' | ||
|
|
||
| # string used to build the filenames of the intermediate scorefiles (one valid date). Only restriction: | ||
| # no other file in the output directory contains that string in the filename | ||
| # example: 'emulator_onPL_ALL_obs_2020' | ||
| expDescr '{{ experiment_description }}' | ||
|
|
||
| # string used to build the filenames of the final scorefiles. | ||
| # please follow the naming convention explained on the wiki page | ||
| # This should match the output file for this from snakemake rule | ||
| # Example: 'exp_ACOSMO-2-models_C-2E-CTRL_2020' | ||
| fileDescr '{{ file_description }}' | ||
|
|
||
|
|
||
| # string to filter feedback file input. All feedback files that contain the string in their filename are used. | ||
| # regular expressions are allowed, for details see list.files() in R | ||
| filePattern 'verSYNOP' | ||
|
|
||
| # switch experiment/routine verification. Set to T. (F only for DWD routine verification) | ||
| experiment 'T' | ||
|
|
||
| # switch for testing if two model versions differ significantly (T/F). Set to T only if there are two or more model versions to verify | ||
| sigTest 'T' | ||
|
|
||
| # subdomains to verify | ||
| subdomains 'ALL,USER' | ||
| # Location of domain table and blacklists. This data should be mounted from balfrin into | ||
| # the container. Recommend using the same path in src/dest, for simplicity. | ||
| # Example: '/users/paa/01_store/02_FFV2/data/7_ML_inner_polygon' | ||
| domainTable '{{ domain_table }}' | ||
| # Example: '/users/paa/01_store/02_FFV2/data/blacklist' | ||
| blacklists '{{ blacklists }}' | ||
|
|
||
| # time binning steps for valid times (in minutes) relative to the valid time of the feedback file (timestamp in filename) read | ||
| # (time granularity of the verification) | ||
| # does not apply to data of more than one FF (because FF are read sequently) | ||
| timeSteps '0' | ||
|
|
||
| # range of the temporal bins in minutes | ||
| timeBreaks '30,-30' | ||
|
|
||
| # set forecast times for which scores should be calculated. | ||
| # useful filter for lead times with very low LEN. | ||
| #veri_forecast_time '0000,1200,2400,3600,4800,6000,7200,8400,9600,10800,12000' | ||
|
|
||
| # run class(es). See Table 27 in the Feedback File Definition (version of 29 Oct. 2019) | ||
| #veri_run_class '0,2' | ||
|
|
||
| # run type(s). See Table 26 in the Feedback File Definition (version of 29 Oct. 2019) | ||
| veri_run_type '0,4' | ||
|
|
||
| # state(s) of observation to be used. See Table 8 in the Feedback File Definition (version of 29 Oct. 2019) | ||
| #state '0,1,5,7,13' | ||
|
|
||
| # state(s) of report to be used. See Table 8 in the Feedback File Definition (version of 29 Oct. 2019) | ||
| #r_state '0,1,5,7,13' | ||
|
Contributor
There was a problem hiding this comment. Are these changes to the accumulation logic for total precipitation needed here? If not, I would remove these.
Author
There was a problem hiding this comment. Exactly. MEC needs precip accumulated from the beginning of the run
Contributor
There was a problem hiding this comment. There is a global postprocessor defined that is applied to all outputs just above. So the specific postprocessors for accumulation of precipitation introduced here are redundant. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| !============================================================================== | ||
| ! namelist template for MEC | ||
| !============================================================================== | ||
|
|
||
| !=================== | ||
| ! general parameters | ||
| !=================== | ||
| &run | ||
| method = 'GMESTAT' ! Model Equivalent Calculator | ||
| model = 'ML' ! forecast model. One of "COSMO" "ICON" "ML" | ||
| input = './input_mod' ! input data path | ||
| data = '/oprusers/osm/opr.inn/data/' ! data path for auxiliary data | ||
| obsinput = './input_obs' ! observation input data path | ||
| output = '.' ! output data to working directory | ||
| time_ana = {{ init_time }}00 ! analysis date YYYYMMDDHHMMSS | ||
| read_fields = 'ps u t v q geof t2m td2m u_10m v_10m' | ||
| grib_edition = 2 | ||
| grib_library = 2 ! GRIB-API used: 1=GRIBEX 2=GRIB2-API | ||
| cosmo_refatm = 2 ! reference atmosphere to be used for COSMO:1or2 | ||
| fc_hours = 0 ! Default is 3h. Has to be set to 0 if one wants to verify +0h leadtime | ||
| grid_file = '/oprusers/osm/opr.inn/data/grid_descriptions/icon_grid_0001_R19B08_mch.nc' ! grid description file for ICON (needed for REA-L) | ||
| nproc1 = 1 | ||
| nproc2 = 1 | ||
| / | ||
|
|
||
| !=============================== | ||
| ! observation related parameters | ||
| !=============================== | ||
| &observations | ||
| !--------------------------------------------------- | ||
| ! read from CDFIN files (if not set use mon/cof/ekf) | ||
| !--------------------------------------------------- | ||
| read_cdfin = F ! (F): dont read COSMO CDFIN files get obs from ekf | ||
| vint_lin_t = T ! linear vertical interpolation for temperature | ||
| vint_lin_z = T ! linear vertical interpolation for geopotential | ||
| vint_lin_uv = T ! linear vertical interpolation for wind | ||
| ptop_lapse = 850. | ||
| pbot_lapse = 950. | ||
| ! int_nn = T ! horizontal interpolation: nearest neighbor | ||
| / | ||
|
|
||
| !==================== | ||
| ! Ensemble parameters | ||
| !==================== | ||
| &ENKF | ||
| k_enkf = 0 ! ensemble size (0 for det. run) | ||
| det_run = 1 ! set to 1 for deterministic run, 0 for ensemble | ||
| / | ||
|
|
||
| !================================ | ||
| ! Verification related parameters | ||
| !================================ | ||
| &veri_obs | ||
| obstypes = "SYNOP" ! "SYNOP TEMP" | ||
| fc_times = {{ leadtimes }} ! forecast lead time at reference (hhmm) 0000,1200,2400,... | ||
| prefix_in = 'obs' ! prefix for input files. ekf or mon (or obs if a verSYNOP.nc file is used) | ||
| prefix_out = 'ver' | ||
| rm_old = 2 ! overwrite entries in verification file ? | ||
| fc_file = '_FCR_TIMEMM_.grib' ! template for forecast file name | ||
| time_range = 1 | ||
| ekf_concat = F | ||
| ref_runtype = 'any' ! accept any runtype for the reference state | ||
| / | ||
|
|
||
| &report | ||
| time_b = -0029 ! (hhmm, inclusive) | ||
| time_e = 0030 ! (hhmm, exclusive) | ||
| / | ||
|
|
||
| &cosmo_obs | ||
| lcd187 = .true. ! use ground based wind lidar obs | ||
| verification_start = -29 ! (min, inclusive) | ||
| verification_end = 30 ! (min, inclusive) | ||
| / | ||
| &synop_obs | ||
| version = 1 | ||
| / |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any chance we can integrate the additional mec config in the existing templates instead of having to support an additional template? Or will this break the existing templates when run without the mec / ffv2 flags?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Integrating the mec/ffv2 part in to an existing config should be possible. It is a design choice I guess. I have a slight preference for a larger number but small configs but I could try integration. Which existing one would then be best suited?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we are supporting three forecaster configs (I don't know why we even have these 3 and they seem all a bit outdated). So I would have suggested to integrate it with one (or all) of these. But honestly, now that I look at the config mess, I am not even sure this is the way to go...