Enhance Jenkinsfile with status tracking and error handling

[arbetter]

Added environment variables for status tracking and updated error handling in multiple stages. Enhanced logging and commit messages for ORT results.

Commit Queue Requirements:

• This PR addresses a relevant WM issue (if not, create an issue).
• All subcomponent pull requests (if any) have been reviewed by their code managers.
• Run the full Intel+GNU RT suite (compared to current baselines), preferably on Ursa (Derecho or Hercules are acceptable alternatives). Exceptions: documentation-only PRs, CI-only PRs, etc.
  • Commit log file w/full results from RT suite run (if applicable).
  • Verify that test_changes.list indicates which tests, if any, are changed by this PR. Commit test_changes.list, even if it is empty.
• Transparency in the use of generative AI is required by NOAA policy. Was GenAI used in this work?
  • [Generative AI tool (insert name, if any)] was used to assist with developing this code. The code has been reviewed, edited, and validated by NWS staff.
• Fill out all sections of this template.

---

Description:

Provides information in the commit message about whether the run succeeded or failed and the directories for the regional_control and control_p8 ORT tests. Restructures the code so that bash shells are within groovy commands that capture the error status and convert it to build and stage results -- set to "unstable" if the OpnReqTest failed (yellow exclamation point) -- but the build continues to the next stage. If one test fails, the result is the OpnReqTest fails. Pass or fail, that message is included in the Git Commit message as well as the directories for the tests. The run continues to the end and the labels are removed, preventing a retest.

Commit Message:

* UFSWM - Enhance Jenkinsfile with status tracking and error handling
  * AQM - 
  * CATChem - 
  * CDEPS - 
  * CECE - 
  * CICE - 
  * CMEPS - 
  * CMakeModules - 
  * UFSATM - 
    * ccpp-physics -
      * CCPP submodules (list) - 
    * atmos_cubed_sphere -
    * MPAS
  * GOCART -
  * LM4 - 
  * MOM6 - 
  * NOAHMP - 
  * WW3 - 
  * fire_behavior
  * stochastic_physics - 

Priority:

• Critical Bugfix: Reason
• High: User request PSD-143
• Normal

Git Tracking

UFSWM:

• Closes # PSD-143

Sub component Pull Requests:

• None

UFSWM Blocking Dependencies:

• Blocked by #
• None

Documentation:

• Documentation update required.
  • Relevant updates are included with this PR.
  • A WM issue has been opened to track the need for a documentation update; a person responsible for submitting the update has been assigned to the issue (link issue).
• Documentation update NOT required.
  • Explanation: CI PR

---

Changes

Regression Test Changes (Please commit test_changes.list):

• PR Adds New Tests/Baselines.
• PR Updates/Changes Baselines.
• No Baseline Changes.

Input data Changes:

• None.
• PR adds input data.
• PR changes existing input data.

Library Changes/Upgrades:

• Required
  • Library names w/versions:
  • Git Stack Issue (JCSDA/spack-stack#)
• No Updates

---

Testing Log:

• RDHPCS
  • Orion
  • Hercules
  • GaeaC6
  • Derecho
  • Ursa
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
• opnReqTest (complete task if unnecessary)

Testing Remarks:

[arbetter]

I'm out the rest of the week on PTO. Back 8/24