Clarify and clean up how verification masking works in METplus config file templates

[gsketefian]

Description

Once PR #1339 is merged, there will be a user-configurable workflow variable in the SRW config file named VX_MASK (see config_defaults.yaml) that specifies the masking region(s) to use for verification, i.e. the region(s) to which to limit obs-forecast matched pairs when calculating verification metrics with METplus.

In METplus, there are 3 different variables that can be specified in config files to specify a verification mask. For example, for GridStat, these are:

GRID_STAT_MASK_GRID
GRID_STAT_MASK_POLY
GRID_STAT_VERIFICATION_MASK_TEMPLATE

In the SRW, in the METplus config template files (parm/metplus/*.conf), these variables appear as

{{metplus_tool_name}}_MASK_GRID
{{metplus_tool_name}}_MASK_POLY
{{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE

where {{metplus_tool_name}} is the jinja2 variable that specifies the name of the METplus tool for which a config file is being generated.

This issue proposes improvements in the way VX_MASK is used to set one or more of these variables in the template files.

For reference, here is a description of each of the three METplus masking variables.

{{metplus_tool_name}}_MASK_GRID

This is described here and here for METplus's GRID_STAT tool. It behaves as follows (but this info should to be verified since the MET/METplus User's Guides are not absolutely clear on the details):

1. This can be set to either FULL, which will cause MET to verify at all points on the forecast grid, or to one or more of the NCEP grid names, which are strings of the form G###, where ### is the 3-digit code of one of the grids on this page. It may also accept a list consisting of FULL and one or more G### values (verify this). Other values are not accepted.
2. The default value is FULL.
3. If this is set to a null value, i.e.

   {{metplus_tool_name}}_MASK_GRID =
   

   then all points are left out of the verification, i.e. no verification is performed (assuming other masks are not specified via {{metplus_tool_name}}_MASK_POLY and/or {{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE.

{{metplus_tool_name}}_MASK_POLY

This is a comma-separated list of (paths to) files that define verification masking regions. See here and here for details (for GRID_STAT).

{{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE

This is described here. This description does not specify what MET variable this METplus variable maps to.

Also, it is not clear what capability this offers that is not already available via {{metplus_tool_name}}_MASK_POLY (and vice-versa).

Solution

Once PR #1339 is merged, vx masking in the SRW will work as follows:

1. The elements of VX_MASK are treated as the base names of .poly files, so the file names are assumed to be VX_MASK[i]}.poly.
2. For each element of VX_MASK, if such a file exists in the parm/metplus directory, then the full path to that file is generated. Otherwise, the file is assumed to exist in the share/met/poly directory of the METplus installation (without checking for its existence), and the full path to such a file is generated.
3. These paths are passed to the template config files, where they are used to set {{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE in some instances and to {{metplus_tool_name}}_MASK_POLY in others.

Note that VX_MASK does not affect the value of {{metplus_tool_name}}_MASK_GRID in the METplus config files. All occurrences of {{metplus_tool_name}}_MASK_GRID in the template config files set it to FULL, i.e.

{{metplus_tool_name}}_MASK_GRID = FULL

Proposed Modifications Involving {{metplus_tool_name}}_MASK_GRID

1. Since the default value of {{metplus_tool_name}}_MASK_GRID in METplus is already FULL, remove lines in the templates that set {{metplus_tool_name}}_MASK_GRID to FULL.
2. Introduce a new user-specifiable boolean in config_defaults.yaml (something like VERIFY_VX_MASKS_ONLY) to set {{metplus_tool_name}}_MASK_GRID to a null value. This is in case the user does not want the verification to be carried out on the full domain and instead wants it run only on the mask(s) specified via {{metplus_tool_name}}_MASK_POLY and/or {{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE. Then the jinja2 code in the template(s) would be:

   {%- if VERIFY_VX_MASKS_ONLY %}
       {{metplus_tool_name}}_MASK_GRID =
   {%- endif %}
   

3. If VX_MASK is empty (the default) AND VERIFY_VX_MASKS_ONLY is set to True, then print out a warning that the verification domain is null and quit SRW experiment generation.

Proposed Modifications Involving {{metplus_tool_name}}_MASK_POLY and {{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE

1. If any of the .poly files formed from the strings in VX_MASK doesn't exist in either the SRW's parm/metplus directory or the METplus installation's share/met/poly directory, print out an error message and exit SRW experiment generation.
2. If the file exists in both locations, print out a warning and decide which one to choose (probably SRW's should take precedence).
3. Allow a path to be specified as an element of VX_MASK so that users don't have to copy their poly files into parm/metplus.
4. Since there is apparently no difference between {{metplus_tool_name}}_MASK_POLY and {{metplus_tool_name}}_VERIFICATION_MASK_TEMPLATE (as far as we know; need to confirm with METplus team), use only one of these variables in the template config files by changing all occurrences to one or the other.

Requirements**

Run the verification on the masks specified via VX_MASK and/or VERIFY_VX_MASKS_ONLY (or whatever we end up calling this new variable).

Acceptance Criteria (Definition of Done)

Verification runs on the expected masks.

[gsketefian]

@mkavulich Just wanted to tag you on this issue regarding how to use VX_MASK in the METplus config files. I also have a discussion open in the METplus repo here that will hopefully help answer some of the questions in the issue description above.