Motivated by the recent CDEPS bug that prevented NEON spinup cases from running past year 100, @wwieder wondered if it would make sense to have a special test suite with some extra tests that are run when updating externals. These would be tests whose main purpose is to cover changes in externals rather than the CTSM code itself. I like this idea, assuming this test suite would have more than just a few tests in it and/or it has some larger / longer tests (e.g., longer than an hour or with a high core-hour cost). (If it just has a few small-ish tests, then we might as well just keep them in the standard aux_clm test suite.)
For baseline comparisons, we could either generate baselines manually from the previous tag whenever we need to run this test suite or not worry about baseline comparisons for these tests.
An initial member of this test suite could be a > 100-year NEON spinup case.
Another possibility that I just thought of – to reduce the number of different test suites we have – is to add these tests to the ctsm_sci test suite. The purpose of that test suite is different, and the times it makes sense to run it are also different, but there may be some overlap, and maybe (significant) externals updates are about the right frequency at which we should be running the ctsm_sci test suite.
Motivated by the recent CDEPS bug that prevented NEON spinup cases from running past year 100, @wwieder wondered if it would make sense to have a special test suite with some extra tests that are run when updating externals. These would be tests whose main purpose is to cover changes in externals rather than the CTSM code itself. I like this idea, assuming this test suite would have more than just a few tests in it and/or it has some larger / longer tests (e.g., longer than an hour or with a high core-hour cost). (If it just has a few small-ish tests, then we might as well just keep them in the standard aux_clm test suite.)
For baseline comparisons, we could either generate baselines manually from the previous tag whenever we need to run this test suite or not worry about baseline comparisons for these tests.
An initial member of this test suite could be a > 100-year NEON spinup case.
Another possibility that I just thought of – to reduce the number of different test suites we have – is to add these tests to the ctsm_sci test suite. The purpose of that test suite is different, and the times it makes sense to run it are also different, but there may be some overlap, and maybe (significant) externals updates are about the right frequency at which we should be running the ctsm_sci test suite.