Add plotting enhancements by justin-richling · Pull Request #454 · NCAR/ADF

justin-richling · 2026-05-14T20:40:54Z

New feature type

New plot and/or plot enhancement

What is this new feature?

Add flexibility for plotting (contour levels, colormap, etc.) based on pressure level and/or plot type (polar vs global)

Update plots for more professional look, ie Zonal and Meridional are very bare boned, lat/lon were missing units, some variables were not getting proper scaling, fit colorbar better to axes and extend if necessary, etc.

NOTE: There is an additional enhancement for 2d line plots for Zonal/Meridional that allows a color map for the 2d variables (See SWCF example below - variable defaults argument: colormap_2d)

Combine Lat/Lon Vector into Lat/Lon code since it was a lot of reused code.

NOTE: the calculation for percent difference is currently off for lat/lon vectors plots, so this map is being ignored and vector plots default back to the two cases on top and difference centered below

Add option in variable defaults config for variable nickname; remove all percent diff args in defaults yaml and hard code in plotting script. The user will still have the option to supply these args in the yaml file if default is not desired.

Add option for old NCL color maps

Add units cleaning method to ensure units are always available, this was not enforced throughout the code base.

Add code to resolve levels for hemisphere and resolve colormap in prep_contour_plot

Core new features:

Different contours based on vertical levels

RESTOM:
  colormap: "RdBu_r"
  contour_levels_range: [-100, 101, 5]
  diff_contour_range: [-10, 10, 0.5]
  polar_map:
    colormap:
      nh: "RdBu_r"
      sh: "terrain"
    contour_levels_range:
      #nh: [-100, 101, 5]
      sh: [-100, 0, 2.5]
   scale_factor: 1
  add_offset: 0
  new_unit: "W m$^{-2}$"
  category: "TOA energy flux"
  derivable_from: ['FLNT','FSNT']

SWCF:
  colormap: "Reds"
  polar_map:
    colormap:
      nh: "fakemap"
      sh: "Blues"
  contour_levels_range: [-150, 51, 10]
  diff_colormap: "BrBG"
  diff_contour_range: [-20, 21, 2]
  scale_factor: 1
  add_offset: 0
  new_unit: "Wm$^{-2}$"
  obs_file: "CERES_EBAF_Ed4.1_2001-2020.nc"
  obs_name: "CERES_EBAF_Ed4.1"
  obs_var_name: "toa_cre_sw_mon"
  obs_scale_factor: 1
  obs_add_offset: 0
  category: "TOA energy flux"
  colormap_2d: True

U:
  global_latlon_map:
    colormap: # if not specified, default to viridis for any missing values nested (levs, ie 850) or not
      200: "ncl_default"
      #850: "Spectral"
    contour_levels_range:
      200: [-10, 51, 2.5]
      850: [-10, 31, 2.5]
    diff_contour_range:
      200: [-15, 16, 2]
      850: [-5, 5.1, 0.5]
  polar_map:
    colormap:
      nh: "terrain"
      sh: "fake_cmap"
    contour_levels_range:
      nh:
        200: [-10, 31, 2.5]
        850: [-10, 13, 1]
      sh:
        200: [-10, 51, 5]
        850: [-10, 31, 2.5]
    diff_contour_range:
      200: [-10, 10, 1]
      850: [-5, 5.1, 0.5]
      #nh:
      #  200: [-15, 16, 2]
      #  850: [-5, 5.1, 0.5]
      #sh:
      #  200: [-15, 16, 2]
      #  850: [-5, 5.1, 0.5]
  zonal_mean:
    colormap: "ncl_default"
    contour_levels_linspace: [-10, 30, 15] # These will be applied to all zonal 3d mean plots
    #diff_contour_range: # default to figuring our the contour levels from difference from max/min
  scale_factor: 1
  add_offset: 0
  new_unit: "ms$^{-1}$"
  obs_file: "U_ERA5_monthly_climo_197901-202112.nc"
  obs_name: "ERA5"
  obs_var_name: "U"
  vector_pair: "V"
  vector_name: "Wind"
  category: "State"

Surface_Wind_Stress:
  nickname: "Surface Wind Stress"
  new_unit: "N m$^{-2}$"
  category: "Surface variables"

TAUX:
  new_unit: "N m$^{-2}$"
  vector_pair: "TAUY"
  vector_name: "Surface_Wind_Stress"
  category: "Surface variables"
  scale_factor: -1
  add_offset: 0

TAUY:
  new_unit: "N m$^{-2}$"
  vector_pair: "TAUX"
  vector_name: "Surface_Wind_Stress"
  category: "Surface variables"
  scale_factor: -1
  add_offset: 0

closes #452
closes #425
closes #373

Set conditional based on values for different float formats going into original DataFrame Set number of significant values in adf_web.py to catch unnecessary trailing zeros in decimals. Also move most imports to beginning of script

This reverts commit 0dd3910.

nusbaume

Thanks for all of the code cleanup and plot enhancements @justin-richling! I realize it looks like I have a bazillion comments, but the vast majority should hopefully be easy or trivial to implement. Of course if you do have any questions or concerns with anything I wrote just let me know!

nusbaume · 2026-06-03T02:53:41Z

@@ -0,0 +1,649 @@
+name: adf_npl-2024a


I personally don't think we should have an NPL environment file in the ADF (or LDF). In general it's best to only have the minimum number of packages you need to run all the ADF scripts, which I am guessing is significantly less than what is contained here.

If you need help creating a more up-to-date environment file for the ADF then please let me know, as I would be happy to help work on it. Thanks!

nusbaume · 2026-06-03T03:07:39Z

 !.vscode/extensions.json 
 .history

+#NCL RGB files


Total nit-pick, but I might add a space in the comment here to match the format of all the other comments in this file:

Suggested change

#NCL RGB files

# NCL RGB files

nusbaume · 2026-06-03T03:21:21Z

+            vres["season"] = plot['season']
+            vres["hemi"] = plot['hemi']


Another nit-pick, but I might use the same quotation marks on both sides of the equation here (in case someone who doesn't know python well looks at the code and assumes the behavior is different):

Suggested change

vres["season"] = plot['season']

vres["hemi"] = plot['hemi']

vres['season'] = plot['season']

vres['hemi'] = plot['hemi']

nusbaume · 2026-06-04T02:30:57Z

+                                vres["umdlfld_nowrap"] = umseasons[s]
+                                vres["vmdlfld_nowrap"] = vmseasons[s]
+                                vres["uobsfld_nowrap"] = uoseasons[s]
+                                vres["vobsfld_nowrap"] = voseasons[s]
+                                vres["udiffld_nowrap"] = udseasons[s]
+                                vres["vdiffld_nowrap"] = vdseasons[s]
+                                vres["upctdiffld_nowrap"] = upseasons[s]
+                                vres["vpctdiffld_nowrap"] = vpseasons[s]


I realize this is purely an opinion, but in general I feel that "kwarg"-like dictionary inputs into python functions should only contain metadata (e.g. plot_type), and not required data like the fields being plotted. Given this, I think I would bring in the extra vector data fields as optional function arguments instead of keyword arguments. Of course feel free to push back here if you disagree!

nusbaume · 2026-06-04T02:36:35Z

+                                pf.plot_map_and_save(adfobj, plot_name, case_nickname, base_nickname,
+                                                    [syear_cases[case_idx],eyear_cases[case_idx]],
+                                                    [syear_baseline,eyear_baseline],
+                                                    umseasons[s], uoseasons[s],
+                                                    udseasons[s], upseasons[s],
+                                                    obs=obs, **vres)


If you do accept my suggestion on passing the main data fields as optional arguments, then this function call would look something like this instead:

Suggested change

pf.plot_map_and_save(adfobj, plot_name, case_nickname, base_nickname,

[syear_cases[case_idx],eyear_cases[case_idx]],

[syear_baseline,eyear_baseline],

umseasons[s], uoseasons[s],

udseasons[s], upseasons[s],

obs=obs, **vres)

pf.plot_map_and_save(adfobj, plot_name, case_nickname, base_nickname,

[syear_cases[case_idx],eyear_cases[case_idx]],

[syear_baseline,eyear_baseline],

umseasons[s], uoseasons[s],

udseasons[s], upseasons[s],

vm=vmseasons[s], vo=voseasons[s],

vd=vdseasons[s], vp=vpseasons[s],

obs=obs, **vres)

Then in the plot_map_and_save function itself all of the vx variables would be None in the function argument list.

nusbaume · 2026-06-07T03:10:03Z

-
-        fig, ax = plt.subplots(nrows=3)
-        ax = [ax[0],ax[1],ax[2]]
+        fig, ax = plt.subplots(nrows=3)#figsize=(6,8), 


Remove commented-out code?

nusbaume · 2026-06-07T03:15:23Z

+        plot_kwargs = kwargs.copy()
+        plot_kwargs["colormap_2d"] = use_cmap
+        diff_kwargs = plot_kwargs.copy()
+        diff_kwargs["type"] = "diff"
+        diff_kwargs.pop("norm", None)


Instead of making multiple copies, why not just make one copy that is then modified for the diff and pctdiff plots? The only reason you would need multiple copies if you needed to save diff_kwargs after creating the zonal plot for the difference field.

nusbaume · 2026-06-07T03:16:32Z

-        levs = np.unique(np.array(cp_info['levels1']))
-        levs_diff = np.unique(np.array(cp_info['levelsdiff']))
-        levs_pctdiff = np.unique(np.array(cp_info['levelspctdiff']))
+        # Generate zonal plot:


Meridional plot?

Suggested change

# Generate zonal plot:

# Generate meridional plot:

nusbaume · 2026-06-07T03:18:13Z

+        fig, ax = plt.subplots(figsize=(8,10),nrows=4, constrained_layout=True,
+                               sharey=True, **cp_info['subplots_opt'])

+        levs = np.unique(np.array(levels_sim))


Just noting again that naming this variable to something like contour_levs might help make it obvious that this is not related to vertical levels.

nusbaume · 2026-06-07T03:23:12Z

+        use_cmap = kwargs.get("colormap_2d", False)
+        plot_kwargs = kwargs.copy()
+        plot_kwargs["colormap_2d"] = use_cmap
+        diff_kwargs = plot_kwargs.copy()
+        diff_kwargs["type"] = "diff"
+        diff_kwargs.pop("norm", None)


Can we get away with only one copy of kwargs here as well?

brianpm · 2026-06-16T03:32:34Z

                    height="90%",  # height : 90%
                    loc='lower left',
                    bbox_to_anchor=(1.05, 0.05, 1, 1),
                    bbox_transform=ax2.transAxes,


I think this ax2 needs to be pointed at an axs element

brianpm · 2026-06-16T03:33:15Z

                    height="90%",  # height : 90%
                    loc='lower left',
                    bbox_to_anchor=(1.05, 0.05, 1, 1),
                    bbox_transform=ax3.transAxes,


ax3 should be in axs

brianpm · 2026-06-16T03:33:33Z

                    height="90%",  # height : 90%
                    loc='lower left',
                    bbox_to_anchor=(1.05, 0.05, 1, 1),
                    bbox_transform=ax4.transAxes,


ax4 should be in axs

brianpm · 2026-06-16T03:34:51Z

+        a.set_boundary(circle, transform=a.transAxes)
+        a.gridlines(draw_labels=False, crs=ccrs.PlateCarree(), 
+                    lw=1, color="gray",y_inline=True, 
+                    xlocs=range(-180,180,90), ylocs=range(0,90,10))


Is range(0,90,10)going to work for southern hemisphere?

brianpm · 2026-06-16T03:35:33Z

                    loc='lower left',
-                    bbox_to_anchor=(1.05, 0, 1, 1),
+                    bbox_to_anchor=(1.02, 0, 1, 1),
                    bbox_transform=ax2.transAxes,


Should ax2 be axs[...]?

brianpm · 2026-06-16T03:35:55Z

+                        height="100%",
+                        loc='lower left',
+                        bbox_to_anchor=(1.02, 0, 1, 1),
+                        bbox_transform=ax3.transAxes,


ax3 maybe axs[...]

brianpm · 2026-06-16T03:42:25Z

+        msg += f" Trying if this an NCL color map"
+
+        url = guess_ncl_url(cmap_pctdiff)
+        locfil = "." / f"{cmap_pctdiff}.rgb"


I think you have to use some pathlib thing here, probably: locfil = Path.cwd() / f"{cmap_pctdiff}.rgb"

brianpm · 2026-06-16T03:47:46Z

+        pctdiff_mag = diff_mag / np.abs(obs_mag) * 100.0
+        #pctdiff_mag = diff_mag / np.abs((mdl_mag + obs_mag)/2) * 100.0
+        pctdiff_mag = pctdiff_mag.where(np.isfinite(pctdiff_mag), np.nan)
+        pctdiff_mag = pctdiff_mag.fillna(0.0)


Is this what we actually want? Divide by zero is replaced by np.nan, but then get set to 0.0. Won't that mean that any nan value ends up looking like a valid result in the plot?

brianpm · 2026-06-16T03:54:47Z

+                    ax[i].quiver(lons[skip], lats[skip], umdlfld[skip], vmdlfld[skip], mdl_mag.values[skip],
+                            transform=ccrs.PlateCarree(), cmap='Reds')
+                    ax[i].quiver(lons[skip], lats[skip], uobsfld[skip], vobsfld[skip], obs_mag.values[skip],
+                            transform=ccrs.PlateCarree(), cmap='Reds')


Both quiver calls are unconditional inside the if i in [0,1] block — so for both i=0 (model panel) and i=1 (baseline panel), it draws model vectors first and then baseline vectors on top. Both panels end up with overlaid arrows.

It seems like the intent is i=0 → model only, i=1 → baseline only. The fix should be:

if i == 0: ax[i].quiver(..., umdlfld[skip], vmdlfld[skip], mdl_mag.values[skip], ...) elif i == 1: ax[i].quiver(..., uobsfld[skip], vobsfld[skip], obs_mag.values[skip], ...)

brianpm · 2026-06-16T04:06:38Z

  scale_factor: 1
  add_offset: 0
  new_unit: ""
-  pct_diff_contour_levels: [-100,-75,-50,-40,-30,-20,-10,-8,-6,-4,-2,0,2,4,6,8,10,20,30,40,50,75,100]


Before this PR, pct_diff_colormap and pct_diff_contour_levels were at the top level of a variable's YAML entry and were found directly in kwargs.

Now:

If the user has a global_latlon_map (or similar) sub-block, they must put pct_diff_colormap inside that block
which means that the previous behavior of top-level placement is silently ignored.

If plot_type is not passed into the function at all, plot_type_dict = {} and there is no way for the user to override hard-coded values that are given in lib/plotting_utils.py:

Line 863: cmap_pctdiff = "PuOr_r"

Line 882: cmap_pctdiff = 'PuOr_r'

Line 896: levels_pctdiff = [-100,-75,-50,-40,-30,-20,-10,-8,-6,-4,-2,0,2,4,6,8,10,20,30,40,50,75,100]

So the PR description's claim that users can still supply these args is only partially true. The configurability is preserved when nested under a plot-type block but silently lost in the flat/top-level case that was standard before.

This probably doesn't matter much, but it complicates customizing plots from the YAML file. Needs documentation at minimum.

brianpm · 2026-06-16T04:13:14Z

+    vmax = kwargs.get("vmax", None)
+    if use_cmap:
+        if kwargs["type"] == "pctdiff":
+            cmap = kwargs.get("pct_diff_colormap", "PuOr")


The hard-coded yaml default was PuOr_r right? Should that be used here?

brianpm

I left a few comments where I saw some inconsistent changes. Looks good overall. My first thought is that we could probably reduce total code, but no need for now.

justin-richling and others added 30 commits November 3, 2022 08:25

Fix accuracy in AMWG tables

0dd3910

Set conditional based on values for different float formats going into original DataFrame Set number of significant values in adf_web.py to catch unnecessary trailing zeros in decimals. Also move most imports to beginning of script

Revert "Fix accuracy in AMWG tables"

1b9e279

This reverts commit 0dd3910.

Update create_climo_files.py

4a51633

Merge branch 'main' of https://github.com/justin-richling/ADF

c839c5b

Merge branch 'main' of https://github.com/justin-richling/ADF

0046996

Merge branch 'NCAR:main' into main

ab1d335

Merge branch 'NCAR:main' into main

af6bda2

Merge branch 'NCAR:main' into main

8492d90

Merge branch 'NCAR:main' into main

0d5a1bb

Merge branch 'NCAR:main' into main

92d3a57

Merge branch 'NCAR:main' into main

c01918d

Merge branch 'NCAR:main' into main

090350d

Merge branch 'NCAR:main' into main

ab2be2a

Merge branch 'NCAR:main' into main

64d5db1

Merge branch 'NCAR:main' into main

6a44157

Merge branch 'NCAR:main' into main

0f1317f

Merge branch 'NCAR:main' into main

04b5216

Merge branch 'NCAR:main' into main

851918c

Merge branch 'NCAR:main' into main

bc8472c

Merge branch 'NCAR:main' into main

fa05eec

Merge branch 'NCAR:main' into main

102a501

Merge branch 'NCAR:main' into main

fd115b5

Merge branch 'NCAR:main' into main

7436f1d

Merge branch 'NCAR:main' into main

056c02d

Merge branch 'NCAR:main' into main

36fec2f

Merge branch 'NCAR:main' into main

42faece

Merge branch 'NCAR:main' into main

8defc50

Merge branch 'NCAR:main' into main

b9fac09

Merge branch 'NCAR:main' into main

7ae1899

Merge branch 'NCAR:main' into main

ebcc4bd

justin-richling added 6 commits April 27, 2026 16:01

Add vector arg to kwargs dict

450bcd9

Remove html file

18ec2c3

Remove test yaml file

2a9294f

Remove old code

0ccca2d

Fix typo

47cc434

Update small line plot feature

f1e6585

justin-richling requested review from brianpm and nusbaume May 14, 2026 20:40

justin-richling added enhancement New feature or request code clean-up Made code simpler and/or easier to read. plotting Related to plot generation config files Concerns the ADF config (YAML) files labels May 14, 2026

justin-richling marked this pull request as draft May 14, 2026 20:45

justin-richling changed the base branch from dev to main May 14, 2026 20:46

justin-richling marked this pull request as ready for review May 14, 2026 20:46

Merge branch 'main' into plotting-enhancements

866aae0

justin-richling mentioned this pull request Jun 2, 2026

Make PR for Plotting Enhancements #452

Closed

nusbaume requested changes Jun 7, 2026

View reviewed changes

brianpm reviewed Jun 16, 2026

View reviewed changes

Conversation

justin-richling commented May 14, 2026

New feature type

What is this new feature?

Core new features:

Uh oh!

nusbaume left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brianpm left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants