Skip to content

Update develop-ref after dtcenter/MET#3316 and dtcenter/MET#3321 - #3205

Merged
JohnHalleyGotway merged 1393 commits into
develop-reffrom
update_develop_8449efef
Jan 31, 2026
Merged

Update develop-ref after dtcenter/MET#3316 and dtcenter/MET#3321#3205
JohnHalleyGotway merged 1393 commits into
develop-reffrom
update_develop_8449efef

Conversation

@github-actions

@github-actions github-actions Bot commented Jan 30, 2026

Copy link
Copy Markdown
Contributor

Remove FULL from the default list of masking regions.
Created by @JohnHalleyGotway

PR dtcenter/MET#3316 for issue dtcenter/MET#3171 modifies the structure of Grid-Diag output files by adding a mask dimension, changing a variable name, and adding new option output for mutual information. This causes 1 diff in the METplus use cases.

PR dtcenter/MET#3321 for issue dtcenter/MET#3298 modified the default behavior of the MET statistics tools. Since grid = [ "FULL" ]; appeared in all the default configuration files, output for the FULL domain was always created, unless mask.grid was explicitly overridden via the METplus conf file.

That issue/PR changed this default behavior by removing "FULL" from the default MET config files. Now, the MET tools check to see if ANY masks were requested by the user. If not, add mask.grid = "FULL".

This change has resulted in many diffs in the METplus Use Case output. Data for the FULL domain is gone unless it has specifically been requested for a use case. I inspected the diffs flagged for 14 of the 69 use case groups in this recent METplus Testing Workflow run and note the following:

  1. diff-use_cases_clouds_0 has diffs in 2 files, both just FULL missing.
  2. diff-use_cases_clouds_1 has diffs in 2 files, both just FULL missing.
  3. diff-use_cases_clouds_2 has diffs in 4 files, all just FULL missing.
> cat `find ./ -name "*_output.txt"` `find ./ -name "_output.stat"` | wc -l
14
> cat `find ./ -name "*_truth.txt"` `find ./ -name "_truth.stat"` | egrep -v FULL | wc -l
14
  1. diff-use_cases_clouds_3 has diffs in 2 files, both just FULL missing.
  2. diff-use_cases_clouds_4 has diffs in 2 files, both just FULL missing.
  3. diff-use_cases_clouds_5 has diffs in 2 files, both just FULL missing.
  4. diff-use_cases_clouds_8 has diffs in 4 files, all just FULL missing.
  5. diff-use_cases_medium_range_3-5_10 has diffs in 2 files, both just FULL missing.
  6. diff-use_cases_medium_range_9 has diffs in 3 files, all just FULL missing.
  7. diff-use_cases_met_tool_wrapper_0-29_59-67 has diffs in 7 files. 6 ensemble_stat output files are just missing FULL, while a grid_diag output file has a expected new variables and dimensions.
  8. diff-use_cases_precipitation_3-7 has diffs in 2 files, both just FULL missing.
  9. diff-use_cases_precipitation_8 has diffs in 21 files, all just FULL missing.
  10. diff-use_cases_s2s_soil_moisture_1 has diffs in 91 files, all just FULL missing.
  11. diff-use_cases_space_weather_0-1 has diffs in 2 files, both just FULL missing.

jprestop and others added 30 commits November 20, 2024 13:52
* update existing builds for upcoming coordinated 6.1 (beta1) release

* added template for existing builds for next dev cycle and added it to ignore list

* Update release guide instructions to set the new location of the Existing Builds page on the Downloads page. Update instructions for creating a new existing builds page to instead copy the template and replace values in the develop branch

* added a step to update the development timeline info for an official release

* Updating existing_builds files

* changed Docker Hub to DockerHub

* added step to update the DockerHub info on the Existing Builds page

* change version to official release because RAL-IT doesn't install development releases on RAL machines

---------

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
* marine_and_cryosphere directories

* fixing underlining

* trying to fix warning msg.

* trying to fix warning msg. again

* Removing space before i.e.

* updating medium_range and pbl directories

* deleting empty file

* removing lots of old text

* precipitation files

* removing junk file

* removing extra #

* add return

* updating 3 dirs s2s, space_weather and unstructured_grids

* updating short_range files

* adding tc_and_extra_tc files

* removing defunct -c option

* added dropdown menus for marine_and_cryosphere

* fixing problems and dropdowns for medium_range

* fixing new problems

* dropdowns for pbl and precip

* adding double colons

* Attempting to resolve errors

* dropdowns for s2s and s2s_mjo

* adding dropdowns for short_range files

* final dropdown directories

* Update docs/use_cases/model_applications/marine_and_cryosphere/GridStat_fcstRTOFS_obsGHRSST_climWOA_sst.py

removing note

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Apply suggestions from code review

adding all of the suggestions

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update GridStat_fcstRTOFS_obsGHRSST_climWOA_sst.py

ensuring literal include is in dropdown menu

* Update GridStat_fcstRTOFS_obsOSTIA_iceCover.py

ensuring literal include is in dropdown

* ensuring literal include is in dropdown menu

* marine_and_cryosphere MET Configuration dropdown fixes

* remaining dropdown directories for MET Configuration

* Julie changing dropdowns

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* HEAD needed to be removed

* There are not MET tools used in this use case

* There are no MET tools used in this use case

* Apply suggestions from code review

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Per #2741, few small changes

* updates from pull request discussion

* removing space

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* fixing dropdown

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* removing bash

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* update from Julie

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Fixing error

* lots of typos

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Adding period

* Adding colon

* Added period

* Added period

* Added period

* typos and more

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

---------

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
* exclude metplus/scripts dir from PyCharm project to prevent incorrect duplicate code warnings with files that are sym linked

* to expand on #2772, updating instructions to include note to set tmp directory for apptainer to prevent issues pulling large images that require a lot of temp space

* per #511, add unit test for expected behavior to support comma-separated lists in a command line single config override that should fail until fix is made

* add a test to ensure that the -c argument is properly ignored since it is can be used in old use cases that were created when the argument was required

* per #511, add support for command line single config overrides to include values that are lists. Simplify logic to parse arguments to strip out -c/--config/-config arguments and skip check/error if argument is invalid because it is already handled in the metplus_config setup step that parses the arguments

* added unit test to ensure that an invalid command line argument causes the appropriate failure from run_metplus.py
* adding In Memoriam section

* Changing to "This Coordinated Release is dedicated to" removing version

* adding links

* fixing Randy's link

* Per #2780, change case of words and add comma after year

---------

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
* per #2816, update title of workflow displayed in Actions tab to display the repo that triggered the workflow on workflow_dispatch events. Preserve the behavior of pull_request and push events by displaying the PR title or head commit message respectively

* fix logic for run-name

* Update event info to note that the workflow was triggered manually if the repo/title was not provided. Update description of workflow_dispatch argument to describe what it can be used for when triggering manually. Only run step to build URL to commit that triggered workflow if it was triggered from an external repo

* add optional argument for workflow_dispatch to specify the title of the workflow run

* revert change to repository argument description since title argument was added to handle custom workflow run titles
* per #2814, add documentation to describe how to set a single config variable on the command line

* fixed typos and formatting issues

* move new content to top of chapter and reword intro sentence

* change label and add link to user env var section

* per #2814, added info and examples for single config overrides with a list of values
* Per #2830, skip closing of log handlers for METplusConfig objects that are created for copying values for process list instances so they are not closed before the end of the run.

* remove some output directories after tests are run
… dev release notes. improve naming of drop downs for generating dev release notes to be consistent
…TC web server to be done for the rc1 release instead of after the official release
* resolve some SonarQube complaints

* per #2586, added function with tests to properly parse list of command line arguments that can now contain comma-separated lists that should not be split up into separate items

* add support for {app}_{data_type}_FILE_WINDOW_BEGIN/END, e.g. GEN_VX_MASK_OBS_FILE_WINDOW_BEGIN. This just adds support for an additional variation of the config variable names

* add support for an empty label for input templates

* update wrapper to be consistent with other wrappers wrt finding input files, progress towards #2492. Allow file window range to be specified separately for mask and input files. Other cleanup to move towards consistent wrappers with fewer wrapper-specific overrides of functions like get_command

* update unit tests to align with changes for #2492

* add documentation for config variables that are newly supported to allow file window range to be specified separately for mask and input files

* renamed GEN_VX_MASK_OBS variables to be GEN_VX_MASK_INPUT as suggested by @JohnHalleyGotway in PR review

* fix logic to properly read input files by handling inputs that support multiple inputs with labels (used by GridDiag and UserScript wrappers) and typical inputs (all other wrappers). Prior to this change only input templates that have the FCST or OBS identifier were read properly via get_input_templates
* Work in progress adding instructions dropdowns to the release guide

* Per #2844, more dropdowns

* Per #2844, add instruction dropdowns for all MET instructions

* Per #2844, more dropdowns.

* Per #2844, use instruction dropdowns throughout

* Per #2844, remove METexpress AWS instruction since its no longer relevant.

* Remove accidentally committed .DS_Store file.
georgemccabe and others added 23 commits December 22, 2025 10:10
* Adding use case image

* Adding MvMODE FireWx use case

* Adding Fire Weather use cases

* Updating use case image

* Adding new use cases to the use case groups

* Updating python embedding file

* Updating documentation

* Updating documentation and MTD use case

* Adding HRRR use case

* Adding documentation for Point-Stat use case

* Updating use case groups, etc

* Fixing some bad documentation links

* Changing m/s to mph

* Fixing weird link

* Fixing file names

* Adding some use case updates

* Updating MvMODE case to use multiple thresholds

* Updating images

* Adding plotting yaml files

* Fixing missing semicolon

* Updating documentation

* Fixing indent

* Turning on one of the tests

* MTD use case testing

* Testing PointStat use case

* Testing MvMODE use case

* Removing CI override because it's behaving oddly

* Updating MvMODE case to run all steps

* Reducing automated testing for fire2

* Adding py_embed to use case group

* Documentation corrections and updates

* Fixing list error

* Another documentation update

* Documentation updates

* Shortening testing for Point-Stat case

* Changes

* Updating contributors guide

* Updating contributors guide

* Updates

* Updating a path to be more consistent

* removing commented out line

* Turning tests off for PR

* SonarQube Sucks

* Updating path

* Fixed incorrect threshold

* updated language, spelling

---------

Co-authored-by: Christina Kalb <kalb@seneca.rap.ucar.edu>
Co-authored-by: j-opatz <jopatz@ucar.edu>
…e or python embedding from check_for_python_embedding function because the function did multiple things and its purpose was confusing. The check_for_python_embedding function now only sets the file_type if the input is a python embedding script, the MET tool supports multiple python inputs, and the file_type was not explicitly specified by the user. The prevents an error because some tools require the file_type to be set for python embedding. Updated unit tests to properly test the two functions.
PR dtcenter/MET#3323 fixed ioda2nc to properly apply this threshold, which causes this setting to wipe out all values and ioda2nc fails. Removing this setting uses the new default value of NA, which should produce the same results as before this bugfix.
@github-project-automation github-project-automation Bot moved this to 🩺 Needs Triage in METplus-13.0 Development Jan 30, 2026
@JohnHalleyGotway JohnHalleyGotway moved this from 🩺 Needs Triage to 🔎 In review in METplus-13.0 Development Jan 30, 2026
@JohnHalleyGotway JohnHalleyGotway added this to the METplus-7.0.0 milestone Jan 30, 2026
@JohnHalleyGotway JohnHalleyGotway changed the title Update develop-ref after dtcenter/MET#3321 Update develop-ref after dtcenter/MET#3316 and dtcenter/MET#3321 Jan 31, 2026
@JohnHalleyGotway JohnHalleyGotway moved this from 🔎 In review to 🏁 Done in METplus-13.0 Development Jan 31, 2026
@JohnHalleyGotway
JohnHalleyGotway merged commit 1753060 into develop-ref Jan 31, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🏁 Done

Development

Successfully merging this pull request may close these issues.