Update to use of ccache in GitHub workflow - #4664
Conversation
Previously, we tried to cache our ccache files, but we were caching the wrong directory! We simplify thinsgs by using an existing action to handle cache for us. We now use it for both the autotools and cmake builds (previously just cmake). We now trigger a new set of builds when a PR is merged into development so we can save the cache. (IF the PR build saved it, then other PR's wouldn't be able to access it.) There's no need to build documentation or run the tests when doing this, so we set a SEED_CACHE_ONLY variable when we do this and use it to skip a bunch of steps We also stop caching the usr-host directory, which was mostly pointless as we primarily use system packages for our dependencies.
d-torrance
added
Infrastructure
|
The idea is to speed up building across different CI runs, right? One way or another, it's assuming that you get allocated the same hardware across different jobs (otherwise you might get weird errors about a wrong CPU command etc). How is this guaranteed? |
Yes, exactly. You can see the caches from a repo's worfklow under the Actions tab, on the left under "Management". Here's ours: https://github.com/Macaulay2/M2/actions/caches Here's the cache on my fork from testing this PR: https://github.com/d-torrance/M2/actions/caches
I suppose, yeah. If that were to happen though, it looks we'd get a cache miss from ccache (see ccache/ccache#725), and just compile as usual. ccache-action is used by more than 10k repos, so I'm guessing this hasn't been an issue. |
that's a different story - what I meant is the scenario when you deploy the same compiler on slightly different CPUs: the compiler knows what commands to emit for each CPU, and potentially a cached result satisfies the constraints to be used, but then you get a runtime error as your resulting executable has an invalid for the CPU command... Or perhaps it's not happening as what's cached is higher level stuff... |
2 participants
Previously, we tried to cache our ccache files, but we were caching the wrong directory!
We simplify things by using an existing action to handle caching for us. We now use it for both the autotools and cmake builds (previously just cmake).
We now trigger a new set of builds when a PR is merged into
developmentso we can save the cache. (If the PR build saved it, then other PR's wouldn't be able to access it.) There's no need to build documentation or run the tests when doing this, so we set aSEED_CACHE_ONLYvariable when we do this and use it to skip a bunch of stepsWe also stop caching the
usr-hostdirectory, which was mostly pointless as we primarily use system packages for our dependencies.Here's a test run:
🤖 AI Disclosure 🤖
Claude wrote the code, but I went over every single line and we ended up making some substantial edits. I also dropped all its extraneous comments lol. I also didn't let it commit and wrote the commit message myself.