Make the MQTT CA certificate configurable

[oysand]

Why

ISAR verified the broker certificate against a CA certificate bundled with the package, at a hardcoded path in mqtt_client.py. Every environment therefore had to share one broker certificate, and the armada integration tests had to be handed the production one.

This is the first of three PRs that remove the need for any external secret to run the armada integration tests. See equinor/armada for the other two.

What

Resolve the CA certificate in this order:

1. ISAR_MQTT_CA_CERT — the CA inline as PEM, written to a temporary file on startup.
2. ISAR_MQTT_CA_CERT_PATH — a path to a CA certificate file.
3. The certificate bundled with ISAR, as today.

Both new settings default to empty, so behaviour is unchanged unless one is set.

The inline form exists because Kubernetes and testcontainers both deliver this kind of material as an environment variable far more easily than as a mounted file. It is what the integration tests use to hand ISAR a CA minted for that test run.

Verified

• 3 new unit tests covering all three resolution paths.
• make check (mypy) clean, make format applied.
• End to end: built into an isar-robot image and run through the full armada integration suite against a broker using a generated certificate — 4 passed.

Ready for review checklist:

• A self-review has been performed
• All commits run individually
• Temporary changes have been removed, like logging, TODO, etc.
• The PR has been tested locally
• A test has been written
• Relevant issues are linked
  • No issue exists; this is part of the work to remove external secrets from the integration tests.
• There is no remaining work from this PR that requires new issues
• The changes do not introduce dead code as unused imports, functions etc.

Merge order

This is one of five PRs that remove the need for any external secret to run the armada
integration tests. They must merge in this order:

1. Let the broker take its credentials at runtime flotilla#2908 — broker takes its credentials at runtime
   Make the MQTT CA certificate configurable #1166 — MQTT CA certificate configurable
   then wait for the :dev images to publish
2. Generate MQTT credentials per test run armada#100 — generate MQTT credentials per test run
3. Stop passing the unused integration test secret flotilla#2918, Stop passing the unused integration test secret isar-robot#402, Stop passing the unused integration test secret sara#488 — stop passing the unused secret
4. armada follow-up — drop the now-unused secret declaration, retire the key vault

Out of order it breaks: armada's reusable workflow currently declares
INTEGRATION_TEST_AZURE_CLIENT_SECRET as required: true and every caller uses @main, so a
caller that stops passing it before step 2 fails immediately. Conversely step 2 needs the new
broker and the configurable ISAR CA already published, because it feeds the broker its
credentials through the environment.

[oysand]

Pushed b391fbb, from a review of this branch.

_resolve_ca_certificate wrote a NamedTemporaryFile(delete=False) on every call and never removed
it, so each MqttClient leaked a certificate file for the lifetime of the process. The inline
certificate is now written once, cached by content, and removed at exit.

Also documented that an inline certificate has to be spilled to disk because the MQTT client takes a
path — the deployments run with readOnlyRootFilesystem: true, and this lands in /tmp, which every
ISAR manifest in equinor/robotics-infrastructure already mounts as an emptyDir. Worth knowing
before someone removes that mount.

Added a test that the certificate is written once and that the cleanup removes it.