Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/renovate.json5
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,21 @@
"depNameTemplate": "anomalyco/opencode",
"versioningTemplate": "semver"
},
{
// uv ships the TLS stack used by every Bazel and DevContainer Python
// tool, so it must track upstream security releases rather than being
// bumped by hand. Both the `uv` and `uvx` entries share one version.
"customType": "regex",
"fileMatch": [
"^tools/lockfiles/uv\\.lock\\.json$"
],
"matchStrings": [
"\"version\":\\s*\"(?<currentValue>[^\"]+)\""
],
"datasourceTemplate": "github-releases",
"depNameTemplate": "astral-sh/uv",
"versioningTemplate": "semver"
},
{
"customType": "regex",
"fileMatch": [
Expand Down
11 changes: 11 additions & 0 deletions src/s-core-devcontainer/.devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,17 @@ FROM buildpack-deps:noble-curl

LABEL dev.containers.features="common"

# Refresh the system trust store before anything else downloads over TLS. The
# base image carries the CA bundle from its own build date, and the
# common-utils feature deliberately runs with `upgradePackages: false`, so a
# distrusted CA would otherwise stay trusted for the life of the image.
# Deliberately unpinned: a pinned trust store is a stale trust store.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates \
&& update-ca-certificates --fresh \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

COPY tools /usr/local/share/score-tools

RUN userdel -f -r ubuntu
14 changes: 12 additions & 2 deletions src/s-core-devcontainer/.devcontainer/bazel-feature/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,23 @@ rm -f "${COPY_TARGET}/devcontainer-features.env" "${COPY_TARGET}/devcontainer-fe
# used by apt-get only inside this script
DEBIAN_FRONTEND=noninteractive

apt-get update

# Refresh the system trust store before the first TLS download. This must come
# before load_feature_versions.sh, which fetches yq over HTTPS, so every
# download in this script validates against a current set of roots. The feature
Comment on lines +33 to +35

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if I run an outdated container image? Should we now delete all container images, which have outdated certificates? This would be in conflict with having long term reproducible builds

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are definitely conflicts between the ideas of "long term reproducible builds" and "security updates". This isn't resolved here. The assumption I make is security issues have higher priority. If in 10 years you load up a container image which contains virus which infects the build system you delivered with, do you want the same infected build or you want to make a fixed version. There is no right answer here, long term reproducible and security are in conflict. If we prioritize on security, we can be surer we can build the software next week when the exploits start coming...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to point out what the consequences are, if we really want to prevent devcontainer images with known security issues.

As far as I can understand your pull request checks that at container build time no revoked CA certificate is used. There is nothing which checks for that in a running container image and I could at the moment download an older container image.

This means we might have to delete old container images each time there is a security update. Or we need a more sophisticated approach, where also older versions might get fixes.

Can we avoid doing such stuff?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point, and I don't have an answer. The only solution I know for avoiding the topic is to go full hermetic builds where all deps are packed into the devcontainer and then no CA is required.

The only way to be sure all of our dependencies are really available (even beyond these CA issues, but what if PyPI moves or removes packages) is to copy them locally, but this results in a massive devcontainer which brings everyone to a stop for hours of updates every day... I don't know of a good way. I already tried once setting up a sidecar container for a pypi/cargo repos with our deps pinned, which score devcontainer references everywhere. This moves the churn out of the devcontainer, but updating dependencies for all modules has to be centralized across the whole score project to make that work (maybe the least bad option I know)

Discuss on Monday in the INFRA round?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can discuss this on monday. Actually @opajonk and myself would prefer less downloads from the internet. We have always unstable CI at Github, because its internet connection is not stable.

# is also usable standalone on a base image this repository does not control,
# so it cannot rely on the Dockerfile having done this already.
# Deliberately not pinned in versions.yaml: a pinned trust store is a stale
# trust store, and it has to be current before versions.yaml can even be read.
apt-get install -y --no-install-recommends ca-certificates
update-ca-certificates --fresh

# Read tool versions + metadata into environment variables
. /usr/local/share/score-tools/internal/devcontainer/load_feature_versions.sh /devcontainer/features/bazel/versions.yaml

ARCHITECTURE=$(dpkg --print-architecture)

apt-get update

# INSTALL CONTAINER BUILD DEPENDENCIES
# Container build dependencies are not pinned, since they are removed anyway after container creation.
apt-get install apt-transport-https -y
Expand Down
14 changes: 12 additions & 2 deletions src/s-core-devcontainer/.devcontainer/s-core-local/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,24 @@ rm -f "${COPY_TARGET}/devcontainer-features.env" "${COPY_TARGET}/devcontainer-fe
# used by apt-get only inside this script
DEBIAN_FRONTEND=noninteractive

apt-get update

# Refresh the system trust store before the first TLS download. This must come
# before load_feature_versions.sh, which fetches yq over HTTPS, so every
# download in this script validates against a current set of roots. The base
# image carries the CA bundle from its own build date and this feature runs
# with `upgradePackages: false`, so a distrusted CA would otherwise persist.
# Deliberately not pinned in versions.yaml: a pinned trust store is a stale
# trust store, and it has to be current before versions.yaml can even be read.
apt-get install -y --no-install-recommends ca-certificates
update-ca-certificates --fresh
Comment on lines +33 to +43

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now the second time I see this. Would it not be enough to have it in the Dockerfile?


# Read tool versions + metadata into environment variables
. /usr/local/share/score-tools/internal/devcontainer/load_feature_versions.sh /devcontainer/features/s-core-local/versions.yaml

ARCHITECTURE=$(dpkg --print-architecture)
KERNEL=$(uname -s)

apt-get update

# Unminimize the image to include standard packages like man pages
bash -c "yes || true" | unminimize
apt-get install -y man-db manpages manpages-dev manpages-posix manpages-posix-dev
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,30 @@ KERNEL=$(uname -s)
# Read tool versions + metadata into environment variables
. /usr/local/share/score-tools/internal/devcontainer/load_feature_versions.sh /devcontainer/features/s-core-local/versions.yaml

# System trust store
#
# The October 2026 eMudhra incident compromised intermediate CA private keys
# chaining to "emSign ECC Root CA - G3"; Ubuntu distrusts that root as of
# ca-certificates 20260816ubuntu0.24.04.1. The install step upgrades the
# package, but image builds run with `--cache-from`, so a layer assembled
# before that upgrade could still be reused. Asserting the outcome here means
# a stale trust store fails CI instead of being published.
distrusted_root_cn="emSign ECC Root CA - G3"
distrusted_root_sha256="86:A1:EC:BA:08:9C:4A:8D:3B:BE:27:34:C6:12:BA:34:1D:81:3E:04:3C:F9:E8:A8:62:CD:5C:57:A3:6B:BE:6B"

# Prove the bundle is readable first; otherwise the absence check below would
# pass simply because nothing could be parsed out of it.
check "validate the system CA bundle is present and parsable" bash -c \
"[ \"\$(openssl crl2pkcs7 -nocrl -certfile /etc/ssl/certs/ca-certificates.crt | openssl pkcs7 -print_certs -noout | grep -c '^subject=')\" -gt 50 ]"
check "validate the compromised ${distrusted_root_cn} is not in the system CA bundle" bash -c \
"! openssl crl2pkcs7 -nocrl -certfile /etc/ssl/certs/ca-certificates.crt | openssl pkcs7 -print_certs -noout | grep -qF '${distrusted_root_cn}'"

# Bazel runs its JVM against this keystore (see the bazel feature's
# /etc/bazel.bazelrc), so it needs the same guarantee as the OpenSSL bundle.
# Matched on fingerprint because the keytool alias is derived from a filename.
check "validate the compromised ${distrusted_root_cn} is not in the Java keystore" bash -c \
"! keytool -list -keystore /etc/ssl/certs/java/cacerts -storepass changeit | grep -qiF '${distrusted_root_sha256}'"

Comment on lines +24 to +47

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it make sense to have this test in a separate file?

shellcheck_lockfile_version="$(/usr/local/share/score-tools/internal/devcontainer/install.py version shellcheck)"
ruff_lockfile_version="$(/usr/local/share/score-tools/internal/devcontainer/install.py version ruff)"
actionlint_lockfile_version="$(/usr/local/share/score-tools/internal/devcontainer/install.py version actionlint)"
Expand Down Expand Up @@ -87,6 +111,10 @@ check "validate opencode is working and has the correct version" bash -c "openco
# uv
check "validate uv is working and has the correct version" bash -c "uv --version | grep '${uv_lockfile_version}'"
check "validate uvx is working and has the correct version" bash -c "uvx --version | grep '${uvx_lockfile_version}'"
# Both ship from one upstream release and one archive. Renovate matches the two
# entries in uv.lock.json separately, so a partial bump would otherwise leave
# them silently pinned to different releases.
Comment on lines +115 to +116

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You sure about the Renovate behavior? My LLM believes this comment is wrong and might mislead someone.

check "validate uv and uvx are pinned to the same release" bash -c "[ '${uv_lockfile_version}' = '${uvx_lockfile_version}' ]"

# pycairo build from source (requires pkg-config and libcairo2-dev)
check "validate pycairo can be built and installed from source" bash -c "pip install --break-system-packages pycairo==1.29.0"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,17 @@ ENV no_proxy=${no_proxy}

LABEL dev.containers.features="common"

# Refresh the system trust store before anything else downloads over TLS. The
# base image carries the CA bundle from its own build date, and the
# common-utils feature deliberately runs with `upgradePackages: false`, so a
# distrusted CA would otherwise stay trusted for the life of the image.
# Deliberately unpinned: a pinned trust store is a stale trust store.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates \
&& update-ca-certificates --fresh \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Unset proxy variables for all login shells
COPY src/s-core-devcontainer/.devcontainer/unset-proxy.sh /etc/bash_completion.d/unset-proxy.sh
COPY tools /usr/local/share/score-tools
Expand Down
6 changes: 3 additions & 3 deletions tools/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ host-side Bazel invocation may require network access to download the binary.
| `bazelisk` | `1.27.0` | Run the Bazel version selected by a repository |
| `buildifier` | `8.2.1` | Format and lint Bazel files |
| `opencode` | `1.18.31` | Run the OpenCode CLI |
| `pre-commit` | `4.5.1` | Run repository pre-commit hooks |
| `pre-commit` | `4.7.0` | Run repository pre-commit hooks |
| `ruff` | `0.11.13` | Check and format Python code |
| `shellcheck` | `0.10.0` | Check shell scripts |
| `starpls` | `0.1.22` | Provide language-server support for Starlark |
| `uv` | `0.10.4` | Manage Python projects and packages |
| `uvx` | `0.10.4` | Run Python tools in isolated environments |
| `uv` | `0.13.5` | Manage Python projects and packages |
| `uvx` | `0.13.5` | Run Python tools in isolated environments |
| `yamlfmt` | `0.17.0` | Format YAML files |

<!-- END GENERATED TOOL TABLE -->
Expand Down
17 changes: 13 additions & 4 deletions tools/internal/refresh_lockfile_digests.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,19 +68,26 @@ def refresh_lockfile(path, token=None):
continue

# Derive the GitHub repo from the first binary URL:
# https://github.com/<owner>/<repo>/releases/download/v<ver>/<asset>
# https://github.com/<owner>/<repo>/releases/download/<tag>/<asset>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe then @FScholPer does not need to fix this script anymore

first_url = tool["binaries"][0].get("url", "")
parts = first_url.split("/")
if len(parts) < 8 or parts[2] != "github.com" or parts[5] != "releases":
if len(parts) < 9 or parts[2] != "github.com" or parts[5] != "releases":
print(
f" ! {tool_name}: cannot derive repo from {first_url!r}",
file=sys.stderr,
)
continue
owner, repo = parts[3], parts[4]

# Release tags are not uniformly prefixed: most projects publish
# `v1.27.0`, while the astral-sh ones (uv, ruff) publish a bare
# `0.13.5`. Reuse the prefix the pinned URL already encodes, because
# Renovate rewrites the version inside that URL but never the prefix.
Comment on lines +84 to +85

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You sure about the Renovate behavior? My LLM believes this comment is wrong and might mislead someone.

# Assuming `v` here silently rewrote those tools to a nonexistent tag.
tag = ("v" if parts[7].startswith("v") else "") + version

release = _api_get(
f"{GITHUB_API}/repos/{owner}/{repo}/releases/tags/v{version}", token
f"{GITHUB_API}/repos/{owner}/{repo}/releases/tags/{tag}", token
)
digests = {a["name"]: a.get("digest", "") for a in release.get("assets", [])}

Expand All @@ -96,7 +103,9 @@ def refresh_lockfile(path, token=None):
sha = digest.split(":", 1)[1] if digest.startswith("sha256:") else digest
# Rebuild the URL from the version so the file is consistent
# whether or not Renovate already updated the URL.
new_url = f"https://github.com/{owner}/{repo}/releases/download/v{version}/{asset}"
new_url = (
f"https://github.com/{owner}/{repo}/releases/download/{tag}/{asset}"
)
if binary.get("sha256") != sha or binary.get("url") != new_url:
binary["url"] = new_url
binary["sha256"] = sha
Expand Down
2 changes: 1 addition & 1 deletion tools/lockfiles/python_tools.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ PYTHON_TOOLS = {
# users. They are separate because Python distributions may name them
# differently.
"package": "pre-commit",
"version": "4.5.1",
"version": "4.7.0",
"entrypoint": "pre-commit",
# Documentation is generated from the catalog so version and purpose
# are reviewed together.
Expand Down
28 changes: 14 additions & 14 deletions tools/lockfiles/uv.lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tools/tests/python_tool_runner_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ chmod +x "${fake_uvx}"
# The explicit assertion verifies the requested release. Subsequent checks use
# the parsed value to keep every expected uv argument aligned with the catalog.
catalog_version="$("${installer}" version pre-commit)"
[[ "${catalog_version}" = "4.5.1" ]]
[[ "${catalog_version}" = "4.7.0" ]]

# The Bazel launcher receives an execroot-relative uvx path, then changes to the
# caller's workspace. This catches regressions where that path breaks after cd.
Expand Down
Loading