Repository navigation
fix: emudhra ca certificate update #175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,14 +30,24 @@ rm -f "${COPY_TARGET}/devcontainer-features.env" "${COPY_TARGET}/devcontainer-fe | |
| # used by apt-get only inside this script | ||
| DEBIAN_FRONTEND=noninteractive | ||
|
|
||
| apt-get update | ||
|
|
||
| # Refresh the system trust store before the first TLS download. This must come | ||
| # before load_feature_versions.sh, which fetches yq over HTTPS, so every | ||
| # download in this script validates against a current set of roots. The base | ||
| # image carries the CA bundle from its own build date and this feature runs | ||
| # with `upgradePackages: false`, so a distrusted CA would otherwise persist. | ||
| # Deliberately not pinned in versions.yaml: a pinned trust store is a stale | ||
| # trust store, and it has to be current before versions.yaml can even be read. | ||
| apt-get install -y --no-install-recommends ca-certificates | ||
| update-ca-certificates --fresh | ||
|
Comment on lines
+33
to
+43
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is now the second time I see this. Would it not be enough to have it in the |
||
|
|
||
| # Read tool versions + metadata into environment variables | ||
| . /usr/local/share/score-tools/internal/devcontainer/load_feature_versions.sh /devcontainer/features/s-core-local/versions.yaml | ||
|
|
||
| ARCHITECTURE=$(dpkg --print-architecture) | ||
| KERNEL=$(uname -s) | ||
|
|
||
| apt-get update | ||
|
|
||
| # Unminimize the image to include standard packages like man pages | ||
| bash -c "yes || true" | unminimize | ||
| apt-get install -y man-db manpages manpages-dev manpages-posix manpages-posix-dev | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,30 @@ KERNEL=$(uname -s) | |
| # Read tool versions + metadata into environment variables | ||
| . /usr/local/share/score-tools/internal/devcontainer/load_feature_versions.sh /devcontainer/features/s-core-local/versions.yaml | ||
|
|
||
| # System trust store | ||
| # | ||
| # The October 2026 eMudhra incident compromised intermediate CA private keys | ||
| # chaining to "emSign ECC Root CA - G3"; Ubuntu distrusts that root as of | ||
| # ca-certificates 20260816ubuntu0.24.04.1. The install step upgrades the | ||
| # package, but image builds run with `--cache-from`, so a layer assembled | ||
| # before that upgrade could still be reused. Asserting the outcome here means | ||
| # a stale trust store fails CI instead of being published. | ||
| distrusted_root_cn="emSign ECC Root CA - G3" | ||
| distrusted_root_sha256="86:A1:EC:BA:08:9C:4A:8D:3B:BE:27:34:C6:12:BA:34:1D:81:3E:04:3C:F9:E8:A8:62:CD:5C:57:A3:6B:BE:6B" | ||
|
|
||
| # Prove the bundle is readable first; otherwise the absence check below would | ||
| # pass simply because nothing could be parsed out of it. | ||
| check "validate the system CA bundle is present and parsable" bash -c \ | ||
| "[ \"\$(openssl crl2pkcs7 -nocrl -certfile /etc/ssl/certs/ca-certificates.crt | openssl pkcs7 -print_certs -noout | grep -c '^subject=')\" -gt 50 ]" | ||
| check "validate the compromised ${distrusted_root_cn} is not in the system CA bundle" bash -c \ | ||
| "! openssl crl2pkcs7 -nocrl -certfile /etc/ssl/certs/ca-certificates.crt | openssl pkcs7 -print_certs -noout | grep -qF '${distrusted_root_cn}'" | ||
|
|
||
| # Bazel runs its JVM against this keystore (see the bazel feature's | ||
| # /etc/bazel.bazelrc), so it needs the same guarantee as the OpenSSL bundle. | ||
| # Matched on fingerprint because the keytool alias is derived from a filename. | ||
| check "validate the compromised ${distrusted_root_cn} is not in the Java keystore" bash -c \ | ||
| "! keytool -list -keystore /etc/ssl/certs/java/cacerts -storepass changeit | grep -qiF '${distrusted_root_sha256}'" | ||
|
|
||
|
Comment on lines
+24
to
+47
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. does it make sense to have this test in a separate file? |
||
| shellcheck_lockfile_version="$(/usr/local/share/score-tools/internal/devcontainer/install.py version shellcheck)" | ||
| ruff_lockfile_version="$(/usr/local/share/score-tools/internal/devcontainer/install.py version ruff)" | ||
| actionlint_lockfile_version="$(/usr/local/share/score-tools/internal/devcontainer/install.py version actionlint)" | ||
|
|
@@ -87,6 +111,10 @@ check "validate opencode is working and has the correct version" bash -c "openco | |
| # uv | ||
| check "validate uv is working and has the correct version" bash -c "uv --version | grep '${uv_lockfile_version}'" | ||
| check "validate uvx is working and has the correct version" bash -c "uvx --version | grep '${uvx_lockfile_version}'" | ||
| # Both ship from one upstream release and one archive. Renovate matches the two | ||
| # entries in uv.lock.json separately, so a partial bump would otherwise leave | ||
| # them silently pinned to different releases. | ||
|
Comment on lines
+115
to
+116
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You sure about the Renovate behavior? My LLM believes this comment is wrong and might mislead someone. |
||
| check "validate uv and uvx are pinned to the same release" bash -c "[ '${uv_lockfile_version}' = '${uvx_lockfile_version}' ]" | ||
|
|
||
| # pycairo build from source (requires pkg-config and libcairo2-dev) | ||
| check "validate pycairo can be built and installed from source" bash -c "pip install --break-system-packages pycairo==1.29.0" | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -68,19 +68,26 @@ def refresh_lockfile(path, token=None): | |
| continue | ||
|
|
||
| # Derive the GitHub repo from the first binary URL: | ||
| # https://github.com/<owner>/<repo>/releases/download/v<ver>/<asset> | ||
| # https://github.com/<owner>/<repo>/releases/download/<tag>/<asset> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe then @FScholPer does not need to fix this script anymore |
||
| first_url = tool["binaries"][0].get("url", "") | ||
| parts = first_url.split("/") | ||
| if len(parts) < 8 or parts[2] != "github.com" or parts[5] != "releases": | ||
| if len(parts) < 9 or parts[2] != "github.com" or parts[5] != "releases": | ||
| print( | ||
| f" ! {tool_name}: cannot derive repo from {first_url!r}", | ||
| file=sys.stderr, | ||
| ) | ||
| continue | ||
| owner, repo = parts[3], parts[4] | ||
|
|
||
| # Release tags are not uniformly prefixed: most projects publish | ||
| # `v1.27.0`, while the astral-sh ones (uv, ruff) publish a bare | ||
| # `0.13.5`. Reuse the prefix the pinned URL already encodes, because | ||
| # Renovate rewrites the version inside that URL but never the prefix. | ||
|
Comment on lines
+84
to
+85
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You sure about the Renovate behavior? My LLM believes this comment is wrong and might mislead someone. |
||
| # Assuming `v` here silently rewrote those tools to a nonexistent tag. | ||
| tag = ("v" if parts[7].startswith("v") else "") + version | ||
|
|
||
| release = _api_get( | ||
| f"{GITHUB_API}/repos/{owner}/{repo}/releases/tags/v{version}", token | ||
| f"{GITHUB_API}/repos/{owner}/{repo}/releases/tags/{tag}", token | ||
| ) | ||
| digests = {a["name"]: a.get("digest", "") for a in release.get("assets", [])} | ||
|
|
||
|
|
@@ -96,7 +103,9 @@ def refresh_lockfile(path, token=None): | |
| sha = digest.split(":", 1)[1] if digest.startswith("sha256:") else digest | ||
| # Rebuild the URL from the version so the file is consistent | ||
| # whether or not Renovate already updated the URL. | ||
| new_url = f"https://github.com/{owner}/{repo}/releases/download/v{version}/{asset}" | ||
| new_url = ( | ||
| f"https://github.com/{owner}/{repo}/releases/download/{tag}/{asset}" | ||
| ) | ||
| if binary.get("sha256") != sha or binary.get("url") != new_url: | ||
| binary["url"] = new_url | ||
| binary["sha256"] = sha | ||
|
|
||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if I run an outdated container image? Should we now delete all container images, which have outdated certificates? This would be in conflict with having long term reproducible builds
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there are definitely conflicts between the ideas of "long term reproducible builds" and "security updates". This isn't resolved here. The assumption I make is security issues have higher priority. If in 10 years you load up a container image which contains virus which infects the build system you delivered with, do you want the same infected build or you want to make a fixed version. There is no right answer here, long term reproducible and security are in conflict. If we prioritize on security, we can be surer we can build the software next week when the exploits start coming...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted to point out what the consequences are, if we really want to prevent devcontainer images with known security issues.
As far as I can understand your pull request checks that at container build time no revoked CA certificate is used. There is nothing which checks for that in a running container image and I could at the moment download an older container image.
This means we might have to delete old container images each time there is a security update. Or we need a more sophisticated approach, where also older versions might get fixes.
Can we avoid doing such stuff?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a good point, and I don't have an answer. The only solution I know for avoiding the topic is to go full hermetic builds where all deps are packed into the devcontainer and then no CA is required.
The only way to be sure all of our dependencies are really available (even beyond these CA issues, but what if PyPI moves or removes packages) is to copy them locally, but this results in a massive devcontainer which brings everyone to a stop for hours of updates every day... I don't know of a good way. I already tried once setting up a sidecar container for a pypi/cargo repos with our deps pinned, which score devcontainer references everywhere. This moves the churn out of the devcontainer, but updating dependencies for all modules has to be centralized across the whole score project to make that work (maybe the least bad option I know)
Discuss on Monday in the INFRA round?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can discuss this on monday. Actually @opajonk and myself would prefer less downloads from the internet. We have always unstable CI at Github, because its internet connection is not stable.