Skip to content

[develop] Updated option to use PLATFORM=container instead of singularity - #1373

Merged
MichaelLueken merged 20 commits into
ufs-community:developfrom
natalie-perlin:feature/platform-container
May 29, 2026
Merged

[develop] Updated option to use PLATFORM=container instead of singularity#1373
MichaelLueken merged 20 commits into
ufs-community:developfrom
natalie-perlin:feature/platform-container

Conversation

@natalie-perlin

@natalie-perlin natalie-perlin commented Apr 30, 2026

Copy link
Copy Markdown
Collaborator

DESCRIPTION OF CHANGES:

  • Updated option for PLATFORM=container for GNU and Intel compilers, to build and run SRW. Both GNU- and Intel-based compilers have Rockylinux9 and include spack-stack-libraries v1.9.2 for the current UFS WM and SRW App.
  • Both containers are staged on NOAA RDHPCs Tier-1 platforms in common locations.
  • GNU-based singularity/apptainer containers could be built on a communty platform from a DockerHub repository image, and include GCC-13.3.1 compiler suite and OpenMPI 4.1.6.
  • Intel container that could be downloaded from a DockerHub repository does not include compilers or MPI. A workaround to build a combined container with all the libraries and compilers is suggested (in the Documentation)

INSTRUCTIONS FOR TESTING

The containerized SRW App community test has been successfully run with both GNU and Intel containers on the following platforms: Ursa, Gaea-c6, Hercules, Orion, NOAA Cloud AWS (Rocky9), and NOAA Cloud Azure (Rocky9).

1. Locate a singularity/apptainer container in a shared location (a) or build a GNU container from a Docker repository (b).

(a) Use existing containers in a shared location. Common locations are as follows:

---------------------------------------------------------------------
|  Machine          |    Container path                             |
---------------------------------------------------------------------
| Ursa              | /scratch3/NCEPDEV/nems/role.epic/containers   |
| Gaea-c6           | /gpfs/f6/bil-fire8/world-shared/containers    |
| Hercules/Orion    | /work/noaa/epic/role-epic/contrib/containers  |
| NOAA Cloud AWS(*) | /contrib/EPIC/containers                      |
| NOAA Cloud Azure  | /contrib/EPIC/containers                      |
---------------------------------------------------------------------

(*) NOAA Cloud AWS and Azure - only using EPIC resources via ParallelWorks, and when the storage is attached to the AWS or Azure cluster used for testing.

For GNU container use the following:
export IMG=<full-container-path>/rocky9-gcc13-ss192-ompi416.sif

For Intel container use the following:
export IMG=<full-container-path>/rocky9-oneapi2024.2-ss192.sif

(b)
Build a GNU singularity (*.sif) container image from a Docker Hub repository. It could be done interactively on a service partition on Ursa, or on login nodes on other systems. A singularity or apptainer module may need to be loaded.
Use singularity or apptainer command depending on the module loaded (apptainer given as an example):

apptainer build rocky9-gcc13-ss192-ompi416.sif docker://noaaepic/rocky9-gcc13.3.1-spack-stack:v1.9.2-ufs-env-ompi416

2. Checkout the SRW feature/platform-container branch and sub-modules:

git clone -b feature/platform-container git@github.com:natalie-perlin/ufs-srweather-app.git
cd ./ufs-srweather-app && ./manage_externals/checkout_externals

3. Shell into the container to build the SRW Application.
Use singularity or apptainer depending on availability on the platform of your choice, as outlined for the Tier 1 platforms below:

Ursa: apptainer (no need to load modules)
Gaea-c6: apptainer (no need to load modules)
Hersules: singularity (module load singularity)
Orion: singularity (module load singularity)
noaacloud: singularity (no need to load modules)

Bind-mount host directories into a container, listing each directory with a preceding "-B" flag. The following directories need to be bind-mounted on each Tier 1 platform:

Ursa: /scratch3, /scratch4
Gaea-c6: /gpfs, /ncrc/home2
Hercules/Orion: /work, /work2, /local
NOAA Cloud AWS, Azure (ParallelWorks): /contrib, /lustre(*)
(*) if staging the run or containers locally on /lustre

When shelled into the container, terminal prompt Apptainer> or Singularity> appears.

For example, shelling into a container on Gaea-c6 and building the SRW App for the gnu container looks as follows:

apptainer shell -B /gpfs -B /ncrc/home2 -e $IMG
./devbuild.sh --bin-dir=bin --platform=container --compiler=gnu 2>&1 | tee log.build.001

Use singularity or apptainer depending on availability, and gnu or intel depending on the container used.

Conda environments will be installed, followed by building of the SRW App. A wrapper script (srw.sh)and
a run-time environment file (ufs-srw.env) are prepared and written to the main ./ufs-srweather-app during the build. When all has been finished, type exit to exit from the interactive container shell

4. Prepare a workflow module file, all located in ./modulefiles directory.
There is a wflow_container.lua file that you may want to copy and keep for reference.
Examples below assume standard platform names are used instead of :
ursa, gaeac6, orion, hercules, noaacloud

On Tier 1 systems,
use and copy a wflow_.lua* module file for the corresponding platform:

cp wflow_<platform>.lua wflow_container.lua

Load the singularity or apptainer module on platforms that need it to use the software. For example, add loading the to wflow_hercules.lua and wflow_orion.lua:

load("singularity")

5. Adapt configuration files for the platform and for the community test.
a Prepare a configuration file:

cd ../ush
cp config.container.yaml config.yaml

Make the following changes in ./ush/config.yaml:
ACCOUNT: epic (set account/project name)
COMPILER: gnu (set compiler to whether gnu or intel)
USE_CRON_TO_RELAUNCH: false (set to true if use of cron is allowed on the machine)
EXPT_SUBDIR: test_community (change the experiment name directory if needed)

b Prepare a machine file:

cd ./machines/
cp <platform>.yaml container.yaml

You could modify the NCORES_PER_NODE in ./ush/machine/container.yaml if needed.
Make the following changes to container.yaml.
a Replace the values of following variables:

RUN_CMD_FCST: srun --mpi=pmi2 -n $nprocs
RUN_CMD_POST:  srun --mpi=pmi2 -n $nprocs
RUN_CMD_PRDGEN:   srun --mpi=pmi2 -n $nprocs
RUN_CMD_UTILS:   srun --mpi=pmi2 -n $nprocs

Adapt the variable SCHED_NATIVE_CMD as follows:
On Gaea-c6: SCHED_NATIVE_CMD: --clusters=c6.
Remove the line with this variable from the container.yaml for Hercules, Orion, Ursa, NOAA Cloud.

Note: AQM, NEXUS have not been tested with the container

6. Generate a workflow and launch the workflow:

cd <./ufs-srweather/app>
module use $PWD/modulefiles
module load wflow_container
cd ./ush
./generate_FV3LAM_wflow.py

Export EXPTDIR as suggested in the prompt and start the test, e.g.:

export EXPTDIR=</my/experiment/test_community>
cd $EXPTDIR
rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10

Resubmit periodically and recheck the progress with rocoto:

rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10
rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

TESTS CONDUCTED:

A community test has been run on all Tier 1 Platforms and has successfully completed on all except Derecho.
Derecho fails in make_ics /make_lbcs stages.
More container integration work is needed for Derecho and Ursa to utilize fast interconnect/hpc-x.

  • derecho.intel, derecho.gnu (NOT YET WORKING)
  • gaeac6.gnu
  • gaeac6.intel
  • hercules.gnu
  • hercules.intel
  • orion.gnu
  • orion.intel
  • ursa.gnu
  • ursa.intel
  • NOAA Cloud (indicate which platform) AWS Rocky8 gnu+intel
  • NOAA Cloud (indicate which platform) AWS Rocky9 gnu+intel
  • NOAA Cloud (indicate which platform) Azure Rocky9 gnu+intel
  • Jenkins
  • fundamental test suite
  • comprehensive tests (specify which if a subset was used)

LOGS

logs from rocotostat after test completion:
rocotostat_test_community_ursa_gnu.txt
rocotostat_test_community_ursa_intel.txt
rocotostat_test_community_gaeac6_gnu_2.txt
rocotostat_test_community_gaeac6_intel_2.txt
rocotostat_test_community_hercules_gnu_2.txt
rocotostat_test_community_hercules_intel_2.txt
rocotostat_test_community_orion_gnu_2.txt
rocotostat_test_community_orion_intel_2.txt
rocotostat_test_community_noaacloud_aws_rocky9_gnu.txt
rocotostat_test_community_noaacloud_aws_rocky9_intel.txt
rocotostat_test_community_noaacloud_aws_rocky8_gnu.txt
rocotostat_test_community_noaacloud_aws_rocky8_intel.txt
rocotostat_test_community_noaacloud_azure_gnu.txt
rocotostat_test_community_noaacloud_azure_intel.txt

DEPENDENCIES:

DOCUMENTATION:

Documentation is being prepared, and will be included with this PR asap.

ISSUE:

NB: A PR in progress to the NOAA-RDHPCS Documentation (https://docs.rdhpcs.noaa.gov/), which uses a link to a Chapter on Containers: https://docs.rdhpcs.noaa.gov/software/containers/ . The page updated when the PR is merged: NOAA-RDHPCS/noaa-rdhpcs.github.io#741

So this PR is not a dependency, but rather informational supplement to the SRW documentation that contains a link.

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation.
  • My changes do not require updates to the documentation (explain).
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

  • Work In Progress
  • bug
  • enhancement
  • documentation
  • release
  • high priority
  • run_ci
  • run_we2e_fundamental_tests
  • run_we2e_comprehensive_tests
  • help wanted

CONTRIBUTORS (optional):

@MichaelLueken MichaelLueken added documentation Improvements or additions to documentation enhancement New feature or request Work in Progress labels May 7, 2026
Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst Outdated
@MichaelLueken

Copy link
Copy Markdown
Collaborator

@natalie-perlin -

While attempting to build the GNU container on Ursa, I encounter the following:

-- Install configuration: "RELEASE"
-- Installing: /scratch4/NCEPDEV/stmp/Michael.Lueken/ufs-srweather-app/ursa/bin/build_settings.yaml
Final step for PLATFORM=container is to link executables to a wrapper script
build_settings.yaml
cp: preserving permissions for ‘/scratch4/NCEPDEV/stmp/Michael.Lueken/ufs-srweather-app/ursa/exec/./build_settings.yaml’'build_settings.yaml' -> '/scratch4/NCEPDEV/stmp/Michael.Lueken/ufs-srweather-app/ursa/exec/./build_settings.yaml'
: Invalid argument

All of the executables are in the bin/ directory, but they are never copied to the exec/ directory, so the community test fails because the scripts are unable to find the executables.

@natalie-perlin

Copy link
Copy Markdown
Collaborator Author

@natalie-perlin -

While attempting to build the GNU container on Ursa, I encounter the following:

-- Install configuration: "RELEASE"
-- Installing: /scratch4/NCEPDEV/stmp/Michael.Lueken/ufs-srweather-app/ursa/bin/build_settings.yaml
Final step for PLATFORM=container is to link executables to a wrapper script
build_settings.yaml
cp: preserving permissions for ‘/scratch4/NCEPDEV/stmp/Michael.Lueken/ufs-srweather-app/ursa/exec/./build_settings.yaml’'build_settings.yaml' -> '/scratch4/NCEPDEV/stmp/Michael.Lueken/ufs-srweather-app/ursa/exec/./build_settings.yaml'
: Invalid argument

Michael, thank you for the comments above. It looks like the errors from the build step, after shelling into the container. When you shelled into the container. It looks like something to do with naming of the "--bin-dir" specified in the devbuild.sh line. What was the command line you used to build the SRW App? I may need to include more cases for naming conventions...

As to the exec directory, the links are created by the devbuild.sh script in the updated version, and there is no need for users to perform a separate step.

I have retested the build that had the following:

./devbuild.sh --bin-dir=bin --platform=container --compiler=gnu 

And the final step of the build was as follows:

...
-- Install configuration: "RELEASE"
-- Installing: /scratch3/NCEPDEV/nems/Natalie.Perlin/SRW/ufs-srweather-app/bin/build_settings.yaml
Final step for PLATFORM=container is to link executables to a wrapper script
build_settings.yaml
'build_settings.yaml' -> '/scratch3/NCEPDEV/nems/Natalie.Perlin/SRW/ufs-srweather-app/exec/./build_settings.yaml'
chgres_cube
cpld_gridgen
emcsfc_ice_blend
emcsfc_snow2mdl
filter_topo
fregrid
fvcom_to_FV3
global_cycle
global_equiv_resol
inland
lakefrac
make_hgrid
make_solo_mosaic
ocean_merge
orog
orog_gsl
regional_esg_grid
sfc_climo_gen
shave
ufs_model
upp.x
vcoord_gen
weight_gen
Apptainer> 

@natalie-perlin

Copy link
Copy Markdown
Collaborator Author

@MichaelLueken - minding your last comment, I adapted the cp command to include -v flag only, instead of -pv in devbuild.sh script line 626:

cp -v $file ${SRW_DIR}/exec/.

@MichaelLueken

Copy link
Copy Markdown
Collaborator

@natalie-perlin -

Thanks for checking the devbuild.sh script and updating the cp command to only include the -v option. I was able to successfully build the containerized SRW on Ursa and run the container experiment configuration:

       CYCLE                    TASK                       JOBID               STATE         EXIT STATUS     TRIES      DURATION
================================================================================================================================
201906151800               make_grid                    14062838           SUCCEEDED                   0         1          24.0
201906151800               make_orog                    14062984           SUCCEEDED                   0         1          45.0
201906151800          make_sfc_climo                    14063090           SUCCEEDED                   0         1          42.0
201906151800           get_extrn_ics                    14062839           SUCCEEDED                   0         1         342.0
201906151800          get_extrn_lbcs                    14062840           SUCCEEDED                   0         1         353.0
201906151800         make_ics_mem000                    14063970           SUCCEEDED                   0         1          37.0
201906151800        make_lbcs_mem000                    14063971           SUCCEEDED                   0         1          50.0
201906151800         run_fcst_mem000                    14071816           SUCCEEDED                   0         1        2231.0
201906151800    run_post_mem000_f000                    14073744           SUCCEEDED                   0         1          23.0
201906151800    run_post_mem000_f001                    14073743           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f002                    14073745           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f003                    14073746           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f004                    14073747           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f005                    14073748           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f006                    14073749           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f007                    14073750           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f008                    14073751           SUCCEEDED                   0         1          22.0
201906151800    run_post_mem000_f009                    14073752           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f010                    14073753           SUCCEEDED                   0         1          24.0
201906151800    run_post_mem000_f011                    14073754           SUCCEEDED                   0         1          22.0
201906151800    run_post_mem000_f012                    14073755           SUCCEEDED                   0         1          22.0
201906151800    integration_test_mem000                    14073756           SUCCEEDED                   0         1          11.0

The fundamental WE2E tests were run on Gaea C6 and all tests successfully passed:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
grid_RRFS_CONUScompact_25km_ics_RRFS_lbcs_RRFS_suite_RRFS_sas_202  COMPLETE              15.90
custom_ESGgrid_Central_Asia_3km_20260526093103                     COMPLETE              47.65
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v17_p8_plot  COMPLETE              20.07
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR_2026052  COMPLETE              47.56
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0_20260526093  COMPLETE              23.20
grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_GFS_v16_2026052609315  COMPLETE              34.24
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             188.62

The UFS Fire WE2E tests successfully passed on Gaea C6:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
UFS_FIRE_multifire_two-way-coupled_20260526100716                  COMPLETE              18.03
UFS_FIRE_one-way-coupled_20260526100727                            COMPLETE              25.07
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE              43.10

The AQM WE2E test successfully passed on Gaea C6:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
aqm_grid_AQM_NA13km_suite_GFS_v16_20260526103111                   COMPLETE            2891.44
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE            2891.44

@MichaelLueken MichaelLueken left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natalie-perlin -

I was able to get the executables to successfully copy from the bin/ directory to the exec/ directory. The container experiment successfully passed on Ursa. I ran the fundamental, UFS Fire, and AQM WE2E tests on Gaea C6.

The path to the machine files should be:

ush/machine/*.yaml

I have marked the two locations where machines is used rather than machine. Once these changes have been made, I will approve these changes.

Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst Outdated
Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst Outdated
@natalie-perlin

natalie-perlin commented May 26, 2026

Copy link
Copy Markdown
Collaborator Author

@natalie-perlin -

I was able to get the executables to successfully copy from the bin/ directory to the exec/ directory. The container experiment successfully passed on Ursa. I ran the fundamental, UFS Fire, and AQM WE2E tests on Gaea C6.

The path to the machine files should be:

ush/machine/*.yaml

I have marked the two locations where machines is used rather than machine. Once these changes have been made, I will approve these changes.

Thank you, Michael, for finding this elusive typo! Corrected it, rebuilt the html docs successfully, and pushed the updates to the PR.

@MichaelLueken MichaelLueken left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @natalie-perlin!

Approving PR now.

Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst Outdated
Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst Outdated
Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst Outdated
Comment thread doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst
@MichaelLueken MichaelLueken added the run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests label May 28, 2026
@MichaelLueken

Copy link
Copy Markdown
Collaborator

Several verification WE2E tests have failed on Ursa GNU and Ursa Intel due to issues encountered with the service nodes while attempting to retrieve data from HPSS. I will be resubmitting the failed jobs and will add the logs to the PR once they successfully complete.

@MichaelLueken

Copy link
Copy Markdown
Collaborator

Two tests failed while running the WE2E coverage tests on Azure:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
grid_RRFS_CONUScompact_25km_ics_RRFS_lbcs_RRFS_suite_RRFS_sas_202  COMPLETE              21.51
deactivate_tasks_20260528153148                                    COMPLETE               2.64
specify_template_filenames_20260528153219                          DEAD                  12.57
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_RAP_20260528153  COMPLETE              34.89
grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR_20260528153406  DEAD                  32.19
grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0_202605  COMPLETE              38.27
----------------------------------------------------------------------------------------------------
Total                                                              DEAD                 142.07

Resubmitting specify_template_filenames and grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR WE2E tests.

@MichaelLueken

Copy link
Copy Markdown
Collaborator

Jenkins tests have successfully completed on Derecho, Gaea C6, Hercules, Orion, and PW AWS.

@MichaelLueken

Copy link
Copy Markdown
Collaborator

The two Azure WE2E coverage tests that previously failed have passed during the retest:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used
----------------------------------------------------------------------------------------------------
specify_template_filenames_20260528194300                          COMPLETE              19.27
grid_RRFS_AK_3km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR_20260528194351  COMPLETE             282.56
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             301.83

Thus, Jenkins testing has successfully completed for Azure.

@MichaelLueken

Copy link
Copy Markdown
Collaborator

The WE2E coverage tests have successfully passed for Ursa GNU:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
custom_ESGgrid_Central_Asia_3km_20260528133757                     COMPLETE              71.91
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_nemsio_2019061200_202605  COMPLETE              15.67
get_from_NOMADS_ics_FV3GFS_lbcs_FV3GFS_20260528133824              COMPLETE              23.96
grid_RRFS_CONUS_13km_ics_FV3GFS_lbcs_FV3GFS_suite_HRRR_2026052813  COMPLETE              61.12
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0_20260528133  COMPLETE              30.01
long_fcst_20260528133910                                           COMPLETE             155.72
MET_verification_only_vx_20260528133927                            COMPLETE               0.25
2019_halloween_storm_20260528133945                                COMPLETE              73.65
vx-det_long-fcst_custom-vx-config_aiml-fourcastnet_20260528134003  COMPLETE               0.57
vx-det_long-fcst_custom-vx-config_aiml-panguweather_2026052813402  COMPLETE               0.57
vx-det_long-fcst_custom-vx-config_gfs_20260528134040               COMPLETE               0.69
vx-det_long-fcst_winter-wx_SRW-staged_20260528134059               COMPLETE               7.58
vx-det_multicyc_fcst-overlap_ncep-hrrr_20260528134124              COMPLETE              11.16
vx-det_multicyc_last-obs-00z_ncep-hrrr_20260528134147              COMPLETE              10.04
grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0_202605  COMPLETE              54.89
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             517.79

@MichaelLueken

Copy link
Copy Markdown
Collaborator

The WE2E coverage tests have successfully completed on Ursa Intel:

----------------------------------------------------------------------------------------------------
Experiment name                                                  | Status    | Core hours used 
----------------------------------------------------------------------------------------------------
2019_memorial_day_heat_wave_20260528133709                         COMPLETE              44.43
custom_ESGgrid_Peru_12km_20260528133723                            COMPLETE              26.88
get_from_HPSS_ics_FV3GFS_lbcs_FV3GFS_fmt_grib2_2019061200_2026052  COMPLETE              10.38
get_from_HPSS_ics_GDAS_lbcs_GDAS_fmt_netcdf_2022040400_ensemble_2  COMPLETE             113.66
get_from_HPSS_ics_HRRR_lbcs_RAP_20260528133809                     COMPLETE              24.46
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16_plot_20  COMPLETE              19.85
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_RAP_20260528133842  COMPLETE              17.03
grid_RRFS_CONUScompact_3km_ics_HRRR_lbcs_RAP_suite_HRRR_202605281  COMPLETE             485.92
MET_ensemble_verification_only_vx_time_lag_20260528133927          COMPLETE               4.68
pregen_grid_orog_sfc_climo_20260528133957                          COMPLETE              12.08
smoke_dust_grid_RRFS_CONUS_25km_suite_RRFS_sas_20260528134015      COMPLETE              69.44
vx-det_long-fcst_custom-vx-config_aiml-graphcast_20260528134037    COMPLETE               0.55
vx-det_multicyc_long-fcst-overlap_nssl-mpas_20260528134057         COMPLETE              25.23
vx-det_multicyc_long-fcst-no-overlap_nssl-mpas_20260528134121      COMPLETE              39.73
vx-det_multicyc_first-obs-00z_ncep-hrrr_20260528134146             COMPLETE               8.03
vx-det_multicyc_no-00z-obs_nssl-mpas_20260528134210                COMPLETE               6.96
vx-det_multicyc_no-fcst-overlap_ncep-hrrr_20260528134235           COMPLETE               9.36
grid_SUBCONUS_Ind_3km_ics_FV3GFS_lbcs_FV3GFS_suite_WoFS_v0_202605  COMPLETE              44.85
----------------------------------------------------------------------------------------------------
Total                                                              COMPLETE             963.52

All WE2E coverage tests have successfully completed. Moving forward with merging this PR now.

@MichaelLueken
MichaelLueken merged commit 01641a5 into ufs-community:develop May 29, 2026
7 checks passed
@github-project-automation github-project-automation Bot moved this from In Review to Done in SRW App's PR Reviewers May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants