Signed ContainerProfile fragment bundles - #63
Open
ConstanzeTU wants to merge 66 commits into
Open
Conversation
…tainerProfile Bring the well-tested pkg/signature core (cosign Signer/Verifier, signature.kubescape.io annotations, VerifyObjectAllowUntrusted) and cmd/sign-object onto the migrated ContainerProfile world. AP/NN deprecation collapses the former AP+NN adapter pair and the two tamper verify methods into a single ContainerProfile adapter and one verifyUserContainerProfile: one fetch, one verify, one R1016 site. Wire the tamper-alert exporter in main.go and gate the user-defined CP overlay on load. Adds enableSignatureVerification config (default off). Flat single-signed-CP behavior — the foundation the fragment-bundle layer builds on.
Add pkg/signature/bundle: multiple independently-signed partial ContainerProfiles (fragments) authored by different parties are each verified against a per-class trust policy (allowed signer identities + allowed spec paths, so a client-class signer cannot inject execs into the server profile), deterministically assembled into one composite (append+dedup, ingress/egress by identifier, class-precedence ordering — order-independent), and bound to the exact admissible leaf set via a Merkle leaf-tree manifest. SignComposite re-signs the assembled result with the cluster key so the R1016 tamper path protects it. Signer identity is the signing public-key fingerprint (key-based) or OIDC subject (keyless). Adds HashSignableContent to expose the canonical leaf digest. Unit tests cover the happy path, order-independence, tamper, untrusted signer, class confinement, and unsigned.
…e cache Resolve a user-defined-profile label that names a bundle by listing its fragments (new ListContainerProfiles client method), verifying + assembling + re-signing them into the authoritative composite, which then flows through the existing verify/tamper gate. A present-but-broken bundle suppresses the single-CP fallback; a tampered fragment raises R1016. Enabled by bundleTrustPolicyPath + bundleSigningKeyPath config (mounted ConfigMap + Secret), loaded in main.go via bundle.LoadTrustPolicy/LoadSigningKey. Adds bundle.SignerID for authoring trust policies. Runtime path unit-tested against the storage mock (happy path, not-a-bundle fallback, tampered→R1016).
Test_29_SignedContainerProfile: a signed user-authored CP loads + enforces, and an unlisted exec fires R0001. Test_31_TamperDetectionAlert: modifying a signed CP's spec in storage without re-signing fires R1016 on reload. Both use the sign-after-roundtrip pattern (sign the storage-normalised form so the signed hash matches what node-agent recomputes on load). Adds the curl-signed-cp deployment fixture.
The fork's mirrormain storage removed ApplicationProfile/NetworkNeighborhood from v1beta1 (AP/NN deprecation), so the carried-over AP/NN adapters + sign-object AP/NN branches no longer compile against it (the image build resolves storage via the go.mod replace to the fork commit, not the published v0.0.290 that still has them). Delete the AP/NN adapters, port the generic signer tests + cmd/sign-object to ContainerProfile (seccomp + rules adapters kept), and fix the CLI help text. Builds + tests pass against the fork storage.
verifyUserContainerProfile ran only on initial load (tryPopulateEntry), so a signed CP tampered AFTER it was cached reloaded through the reconciler without re-verification — no R1016 (Test_31 failure). Add the verify gate to refreshOneEntry, right after the RV fast-skip, so it runs exactly once per CP change and R1016 fires on post-load tampering (deduped per resourceVersion).
Test_38_SignedBundleOverlay: three fragments signed by two different keys (vendor: base; operator: admission ingress + overlay execs) round-trip storage signed, assemble into one enforced composite (union proof: an exec allowed only by the overlay fragment stays quiet while an unlisted exec fires R0001), a fragment tampered in storage without re-signing fires R1016 via reconciler re-assembly, and re-signing it recovers the composite (fresh unlisted exec alerts again, overlay-allowed exec stays quiet). Reconciler now re-assembles bundles on refresh instead of degrading the cached composite to a same-named fragment via the single-CP GET; the composite carries the bundle Merkle root as its ResourceVersion so the RV fast-skip holds while fragments are unchanged. Test chart gains the bundle trust policy (per-class signer fingerprints + allowed spec paths) and the cluster signing key, mounted at /etc/bundle (throwaway CI test keys).
The trust policy ConfigMap (per-class signer fingerprints + allowed spec paths) and cluster signing-key Secret that Test_38 depends on; mounted at /etc/bundle. Force-added: the root .gitignore's unanchored 'node-agent' binary pattern also matches this template directory.
…ripped The storage server's List serves items from its metadata table without loading the payload, so listed fragments carry empty specs. Assembly hashed those and flagged every signed fragment as tampered (spurious R1016, bundle never loaded — first Test_38 run). Use the List only to discover the fragment set and Get each fragment for its full spec; a transient Get failure is operational (retry next tick, no R1016). Adds a spec-stripped-List regression test.
The curl image is alpine/busybox: ls lives at /bin/ls, so the absolute /usr/bin/ls exec failed and produced no exec event — starving the R0001 gate (the bundle itself assembled cleanly: 3 fragments, stable root, no R1016). Exec plain names; id still resolves to /usr/bin/id, the path the overlay fragment allows. Log the first probe's exec result for future diagnosis.
37 was the next free number; 38 left a confusing gap. Function, fixture, labels, and doc references renamed — no behaviour change.
… ignores label selectors The storage server returns every CP in the namespace regardless of the List label selector, so assembleUserBundle assembled ALL class-labeled fragments for ANY user-defined-profile name (observed live: a client workload's profile lookup assembled the server's fragments, shadowing its own flat profile). Filter listed items on the bundle label before fetching/verifying. Regression test uses a selector-ignoring mock List.
…erver normalisation Vendors must ship SIGNED fragments, but the storage server normalises specs on save (deflate, with cluster-configurable collapse settings) — so a signature over the shipped form breaks on ingestion, and offline pre-normalisation is unsound because the collapse configuration is per-cluster. Instead the signer embeds the exact canonical signed content in the signature.kubescape.io/content annotation (base64+gzip; annotations are never mutated by storage): verification binds the embedded bytes, and the bundle layer treats them as the verified source of truth — name, labels (class AND bundle membership, so stored-label flips cannot escalate or cross bundles), spec, and leaf digest all come from the signed content. The stored object is a carrier whose spec drift is irrelevant to the chain. Opt-in via WithEmbedContent / sign-object --embed-content (CLI default: on). Legacy sign-after-roundtrip artifacts keep working unchanged.
The assembled-bundle line (fragments + Merkle root) was debug-only, forcing operators to enable global debug logging — which floods the log — just to observe bundle lifecycle. Log root TRANSITIONS (first assembly, fragment-set changes) at info; unchanged per-tick re-assemblies stay at debug.
…m adversarial review Two independent adversarial reviews of the signing/bundle layer surfaced real issues; fixes with regression tests: - V1/C3 (CRITICAL) trust-policy bypass: signerIdentity trusted the unsigned identity/issuer annotations for keyless, so anyone could spoof a trusted signer by stamping two strings (verification is allow-untrusted → no Fulcio attestation). Now the identity is ALWAYS the fingerprint of the public key the signature verified against; the OIDC branch is removed. - C2/V5 embedded-content decoupling: verification hashed the embedded bytes but never bound them to the carrier, and the flat path then enforced the mutable LIVE spec. Now verify.go binds embedded content to the object's name+namespace (mismatch = tamper), and verifyUserContainerProfile enforces the embedded (verified) spec, not the live one. - H1 malformed embedded content on a signed object is now classified as tamper (ErrSignatureMismatch → R1016 + fail-closed), not swallowed as operational. - H2 DecodeSignatureFromAnnotations rejects non-base64 instead of falling back to raw bytes (which could feed a bare attacker public key). - M1/V8 DoS: bound decompressed embedded content (io.LimitReader, 8MiB) and cap fragments per bundle (64) — assembly runs every reconcile tick. - V3 seccomp confinement: any non-zero seccomp content confines to the class, not just DefaultAction. - V7 bundle R1016 dedup keys on the fragment-set fingerprint (name@RV) so distinct tamper states re-alert instead of being masked. - V2 defense-in-depth: AssembleAndVerify self-checks the Merkle root; enforcement remains continuous per-tick re-verification (the root is a provenance record + external-verifier commitment).
…overy The previous fingerprint-keyed dedup stored a key that the clean-recovery path never cleared (it deleted a stale empty key), so a later tamper whose fragment-set fingerprint recurred — which happens because sign-fragment.sh delete/recreates fragments, resetting resourceVersions — was silently deduped and NO R1016 fired (caught in live retest: tamper detected + fail-closed, but zero alerts). Hold the last-alerted fingerprint per bundle and CLEAR it on clean re-assembly, so distinct tampers alert and a recurring one re-alerts after recovery. Regression test covers the tamper→recover→re-tamper cycle.
…rence) The fingerprint-keyed dedup re-alerted only if the recurring fragment-set fingerprint was cleared by an observed clean tick — fragile when delete/recreate resets resourceVersions to the same values. Switch to an edge-trigger: alert once per OBSERVED clean->tamper transition, re-arm on the next observed clean assembly. Keys on tamper STATE, not fragment identity.
- Remove the on-cluster composite signing key: drop SignComposite, the bundleSigningKey field/config/loader. A bundle composite is assembled from already-verified fragments and trusted in-process (fromBundle bypass in both the populate and refresh paths) instead of being re-signed and re-verified. node-agent holds no signing key. - Verify the trust policy against a root public key compiled into the image (embedded, no cluster-side override). The policy is root-signed; node-agent verifies the signature over the canonical policy content and pins the signer to the embedded root fingerprint, failing closed otherwise. sign-object gains a sign-policy subcommand. - Add adversarial bundle tests: client injection with an untrusted key, cross-class key confusion, signature stripping, certificate-swap identity forgery, unknown class, a rollback/replay marker, and signed-policy round-trip / wrong-signer / tamper / missing-signature cases. No private key exists on the cluster or in the repository.
Rules objects are loaded from every namespace and flattened into one list keyed
by rule ID, so anyone able to create a Rules object anywhere could redefine a
rule — for example shipping R0001 with enabled:false — and silently disable
detection cluster-wide. Signing closes that.
A Rules object may now carry signature.kubescape.io/rule-class:
cluster — contributes cluster-wide (the vendor baseline);
namespace — contributes ONLY in its own namespace, and overrides the cluster
rule with the same ID for pods there.
The class, the namespace and the rules are all inside the signed content, so a
fragment cannot be re-classed, moved to another namespace, or edited without
breaking its signature. The trust policy gains a ruleClasses section naming the
signers permitted per class and the rule IDs each class may set ("*" for any).
When the policy declares no ruleClasses, rule signing is off and behaviour is
unchanged: rules load as before, marked cluster-wide, and the namespace scoping
short-circuits. When it is on, a Rules object that fails admission is dropped
whole (fail closed) with the reason logged.
Rule provenance is carried on Rule as SourceNamespace/ClusterWide, both excluded
from serialisation so the signed content is unaffected. Bindings that name a
rule by id or by name now resolve every variant, so the namespace override
applies on that path too, not only where rule bindings are ignored.
entlein
reviewed
Aug 11, 2026
The harness installed node-agent with no trust policy, so a run against an image carrying the signature feature measured it dormant and said nothing about what it costs. A new signing input turns verification on for the AFTER phase only, so running the same image on both sides isolates the feature: the trust policy is mounted from a pre-signed artifact, the cluster ruleset is re-ingested as a signed cluster-class fragment, and the workload is bound to a signed two-fragment bundle that is re-verified on every reconcile tick. enable-signing.sh aborts the run unless bundles and rule fragments both report enabled with nothing rejected. Without that check a rejected ruleset would leave node-agent with no rules to evaluate, doing less work than the baseline and reporting signature verification as free. Keys under benchmark/signing are published and authenticate nothing.
A vendor could not ship a signed fragment at all: metadata.namespace was part of the signed content, so an artifact was locked to whichever namespace the vendor guessed, and re-signing per customer defeats offline signing. Namespace is no longer signed. Where a fragment applies is decided by where it is installed, which is an RBAC question; what the signature binds is the content and the labels. Bundle membership and fragment class are inside the signed content, so they remain unforgeable, and that is what confines a fragment. Rules now use the same two labels as profiles, so one bundle carries both halves: bundle: redis, fragment-class: base the ContainerProfile bundle: redis, fragment-class: overlay the rules for that bundle A base ruleset applies cluster-wide. An overlay applies to exactly the workloads bound to its bundle, in any namespace, overriding the base rule with the same ID for them and for nobody else. Rule provenance is carried as Rule.Bundle, still excluded from serialisation so the signed content is unaffected, and pods are tracked to their bundle from the user-defined-profile label. Also fixes FieldRequirement round-tripping: MarshalJSON emitted null for an undeclared field while the unmarshallers rejected null, so any Rules object that went through the typed struct — including the blob the signing CLI embeds — was unparseable, and every signed rules fragment declaring profileDataRequired would have been dropped. The component tests move onto pre-signed fixtures, the test chart stops shipping a private key in a Secret, and the benchmark can turn verification on so its cost can be measured.
The first signing run reported +0.7% CPU and -18% memory, and neither number meant what it looked like. No bundle was ever assembled: kubectl create and the daemonset patch were suppressed with || true, so a failure to bind the workload passed silently and the per-tick fragment verification -- the dominant cost -- was absent from the measurement. Those calls now surface their errors, the workload rollout is waited on, and the guard fails the run unless a bundle was actually assembled. The memory drop was an artifact of the after phase restarting node-agent to pick up the config while the before phase kept agents that had been running for ten minutes. The before phase now restarts too when signing is measured.
Binding a workload to a user-defined profile makes node-agent stop profiling that container entirely. The load simulator opens a thousand files a second, so dropping its profiling saves far more than signature verification costs, and a run that binds only the after phase reports signing as a 5% CPU and 15% memory saving. The before phase now binds the same workload to an unsigned flat profile with the content the bundle assembles to, so learning is suppressed on both sides and the only difference left is the per-tick verification.
entlein
reviewed
Aug 11, 2026
| @@ -0,0 +1,762 @@ | |||
| apiVersion: kubescape.io/v1 | |||
There was a problem hiding this comment.
we dont want to duplicate those here, pull them in from a repo or call it a fixture
Author
There was a problem hiding this comment.
Fixed in bd84e02: replaced the 762-line copy of the product default with a minimal synthetic fixture (the benchmark measures signing overhead, not detections) and a header marking it as such. The product default stays in helm-charts; not duplicated here.
entlein
reviewed
Aug 11, 2026
entlein
reviewed
Aug 11, 2026
buildEntry records PodUID only when the pod watch already knows the pod, so a container seen first gets an empty one. The only backfill runs through rebuildEntryFromSources, which refreshOneEntry reaches only when the resourceVersion fast-skip misses — and a bundle composite's resourceVersion is its Merkle root, stable by construction, so for a bundle-bound container the skip fires every tick and the UID is never healed. isContainerTerminated then meets a published status whose ContainerID is still empty, cannot match on (name, uid), and falls through to "absent from the status list, so reaped". The entry is evicted and never re-pended, leaving the container with no projected profile — which silently suppresses every rule declaring a profile dependency, R0001 among them, for the rest of its life. Match on container name alone when no UID was captured; a mismatched UID still skips, and a genuinely terminated container still evicts.
|
few thoughts (didn't read all the code to check):
|
|
all very good questions, THANKS!! I have spent some thought on the namespacesthe namespace should be excluded from a Thought process: vendors cannot know what Collisions vs Additivity(thinking in progress - also the tests are not consistent yet)
Sorting the fragment hashes in the Merkle TreeWhats your preference, what is your concern, what guarantees must we provide? WDYT? |
My concern is that the resulting Merkle tree might be different depending on which order we assemble the fragments - so I just want to be deterministic |
… fallback The user-defined-profile label is resolved as a bundle name when bundle signing is enabled, so an identically named ContainerProfile is shadowed. Pin the three outcomes so the fallback semantics cannot regress silently: - a classical ContainerProfile is still resolved by name when no fragment carries that bundle name; - a classical ContainerProfile is shadowed while verifying fragments of the same name assemble; - an unverifiable fragment set does not fall back to the identically named classical ContainerProfile, and never enforces its own content. Refs #67
|
Review round 1, main findings:
|
Introduce the single global signing state that enforcement keys off: - TrustPolicy.Mode (alert|enforce). A mounted policy is ON and at least alerting; enforce refuses unsigned/unverifiable artifacts. Absent or unknown mode resolves to alert, never to silent, so enforce must be asked for explicitly and omission cannot brick a cluster. - DemoRootFingerprint + IsDemoRoot: the published demo root authenticates nothing, so enforce refuses it unless a real root is mounted. A test pins the constant to the fingerprint of the embedded DefaultRootPublicKeyPEM. No enforcement consumes these yet; wiring follows.
…t under enforce The trust policy was verified against a root key path taken from the unsigned, cluster-mutable node-agent config, so an attacker who could edit that ConfigMap could redirect the anchor to their own key. Resolve the root ONLY from the fixed mount (/etc/bundle/root.pub) or the compiled-in default. - LoadSignedTrustPolicyTrusted resolves the root via ResolveTrustedRootFingerprint (fixed mount or compiled, never a config path), verifies the policy, then GuardRootAnchor refuses the published demo root in enforce mode unless a real root is mounted. - cmd/main.go boots through it; logs enforce vs alert, and warns when anchored to a mounted or demo root. - config.BundleRootKeyPath is now ignored (deprecated); the anchor cannot be redirected by config. - fixedRootFingerprint reuses the same trusted resolver (single source). Refs #69 (config-supplied root path).
…intentionally free) checkEmbeddedBinding binds name only; the previous comment wrongly claimed name+namespace. Namespace is deliberately unsigned so the same vendor artifact installs anywhere, and bundle membership (not placement) scopes a fragment. Refs #69 (namespace binding).
…st_43) Checks in the demo's claim that replacing the trust-anchor ConfigMap requires cluster-admin. Via SubjectAccessReview: an unprivileged workload ServiceAccount is denied update/patch/delete/create on node-agent-bundle-policy, while a system:masters subject is allowed. The primary control remains cryptographic (Test_40 refuses a non-root-signed policy regardless of writer); this pins the defense-in-depth RBAC layer.
Test_43 is taken by Test_43_RelativeOpenPathResolution on another active branch; renumber to avoid a merge-time collision and to match the CT matrix.
A still-validly-signed older fragment could be replayed to widen the effective allowlist (known gap TestAdversarial_RollbackReplay_CurrentlyAccepted). Close it with a monotonic version carried in the signed labels. - LabelVersion (signature.kubescape.io/version) is part of the signed labels, so it cannot be forged; admitFragment parses it from the VERIFIED content and it rides on LeafRef.Version. Absent == 0 (back-compatible). - The cache holds a high-water-mark per (namespace, bundle, class, name). A leaf whose version is below its slot's mark rejects the WHOLE assembly (a replayed old fragment cannot slip in beside current ones) and advances no mark; a clean set advances the marks. This is ErrFragmentRollback, NOT a tamper: no R1016, the workload keeps its last verified composite. - Limitation: marks are in-memory and reset on restart; a rollback in the window right after a restart is not caught. Persistence is a follow-up. Refs #69.
The monotonicity gap is closed by the cache-layer version high-water-mark. AssembleAndVerify stays stateless by design; assert the signed version reaches LeafRef.Version so the cache guard has an unforgeable value to enforce.
verifyUserContainerProfile refused unsigned/unverifiable user profiles based on the standalone EnableSignatureVerification flag. Route that decision through signingEnforced(), which is true when the trust policy is in enforce mode OR the legacy flag is set — so 'enforce' is the single global switch while existing requireSignedObjects deployments keep working. No behaviour change for either existing configuration; enforce mode now also gates flat user profiles. Refs the F3 global-mode consolidation.
…failing closed (F1) Bundle membership was discovered by the UNSIGNED bundle label and one inadmissible object failed the whole bundle closed, so anyone with create rights on containerprofiles in a namespace could disable a signed bundle for every workload bound to it (DoS), logged only as a generic warning. Membership is now authenticated: only fragments signed by a class-trusted signer are members. - AssembleAndVerifyPartial verifies each fragment, SKIPS every inadmissible one (returned in dropped), and assembles the admissible remainder. AssembleAndVerify keeps its fail-closed contract (its callers/tests are unchanged); both share assembleVerified. - The cache classifies each drop against its known-good state (bundleVersions doubles as the set of slots that ever verified). A never-admitted drop is a non-member: skipped, with a bundle-scoped edge-triggered alert (never per object, so a spamming writer cannot flood). A previously-admitted slot that now fails is a member-tamper: fail closed, keep the last verified composite, and R1016 when the failure is a signature mismatch. This is the sound form of the skip design: a non-verifying object's certificate is attacker-suppliable, so tamper-vs-skip cannot be judged from the object alone — only a slot that verified before can be a genuine tamper. Tests: TamperedEmitsR1016 / R1016_ReAlertsAfterRecovery updated to the clean-then-tamper flow (tamper of a KNOWN member); new NonMemberInjection_Skipped pins that an injected object is dropped and the bundle survives. Refs #67, #69.
The two-roots change renamed this to 'trust policy rejected', but it is the string demo.md §11 and Test_40_TrustPolicyFailClosed assert against. Restore it; the specific error (including demo-root refusal) rides in the error field.
- baseline-rules.yaml: replace the 762-line copy of the product default ruleset with a minimal synthetic fixture (the benchmark measures signing overhead, not detections) and mark it as such; the product default lives in helm-charts and must not be duplicated here (PR review: don't duplicate / call it a fixture). - apply the rule-signer inversion (base=user/operator key, overlay=vendor key) to benchmark/signing/trust-policy.json, re-sign with the demo root, and sign the base baseline ruleset with operator.pem instead of vendor.pem — consistent with the bob demo.
When rule signing is on and rules objects exist but none are admitted, the effective ruleset is empty and nothing alerts. Rules cannot fail closed to 'blocking' (they are detections, not a gate), so failing closed means 'no rules' — which must not be silent. Emit an Error every sync (a persistent signal) rather than crash-loop, which would keep the operator from signing the baseline to recover. Refs the F3 global-mode consolidation.
…path Reduce the tutorial-style comments to one-line invariants (why skip vs fail-closed, why the alert dedups per bundle, the rollback/marks limitation), per the codebase's no-narrative-comments preference. No behaviour change.
…lign Test_42 Under F1 a stripped signature on a KNOWN member is refused with keep-last-verified (the workload no longer loses its profile) rather than failing the whole bundle. Add the member name(s) to that warning and update Test_42_SignatureStrippedFragmentRejected to assert the new message; 'fragment is not signed', the member name, and no-R1016 still hold.
Author
Component tests: 38/38 greenhttps://github.com/k8sstormcenter/node-agent/actions/runs/31618764691 — all 38 legs pass at Dispatched through the overlay harness with aligned refs: Security fixes validated by this runBundle membership is now authenticated (was a DoS). Membership was discovered by the unsigned Rollback/replay closed. A monotonic version rides in the signed labels (unforgeable) and the cache holds a high-water-mark per (namespace, bundle, class, name); a replayed older-but-valid fragment is refused without R1016 and the workload keeps its last verified composite. Marks are in-memory and reset on restart — a rollback in the window right after a restart is not caught, tracked as a follow-up. Trust anchor cannot be redirected via config. The policy was verified against a root key path read from the unsigned, cluster-mutable node-agent ConfigMap. The anchor now resolves only from the fixed mount or the compiled-in root. Enforce mode additionally refuses the published demo root unless a real root is mounted, since that key authenticates nothing. Single global signing mode. Filed from the review: #67, #68, #69. Note on the earlier red runsEvery failure in the preceding runs was CI setup flakiness ( |
The policy was read once at startup, so rotating a signer, tightening the mode, or adding ruleClasses had no effect until every node-agent restarted — and because the ConfigMap was mounted with subPath, kubelet never even propagated the change into a running pod. Observed on a live cluster: with ruleClasses present in the ConfigMap, rule signing stayed off and unsigned Rules objects kept loading until a manual rollout. - PolicyReloader polls the mounted artifact, and on a CHANGED artifact verifies it against the trusted root (fixed mount or compiled, never config) and applies it through the existing mutex-guarded SetTrustPolicy/SetBundleConfig setters. - A changed artifact that does not verify is REFUSED and the policy already in force is kept, so a swapped-in policy can neither downgrade nor disable enforcement; it is reported once per distinct content, not every tick. The demo-root refusal under enforce applies on reload too, not just at boot. - The component-test chart mounts the policy ConfigMap as a directory instead of a subPath file, which is what lets kubelet deliver the update at all. Tests cover: a verified change taking effect (ruleClasses enabled with no restart), a wrong-key change refused with nothing applied, and enforce refusing the demo root on reload.
A trust policy that carries ruleClasses turns rule admission on, but when the trust anchor is the published demo root key the admission decision authenticates nothing: anyone holding the published private half can sign a fragment that admits. Warn at boot and again whenever a reloaded policy still meets the condition, so the operator sees that signed rules are not a security boundary until a real root is mounted.
…eset Ten Rules-object archetypes (unsigned, signed, wrong signer, corrupted signature, stored-spec tamper with and without embedded content, overlay with/without bundle, disallowed rule ID, admitted-all-disabled) run through the real sync in scenario sets, signing off and on, and the SyncRules payload is compared against an independent per-archetype oracle including (bundle, ID) dedup semantics. A second test pins the detection-outage backstop log to fire exactly when zero fragments admit while Rules objects exist, and not when a fragment admits with only disabled rules.
… and its recovery Test_45: with a ruleClasses policy in force and only the chart's unsigned baseline present, no fragment admits — the agent logs the detection-outage backstop and an exec R0001 would catch stays silent, proving the outage is real and fail-closed. Ingesting the signed baseline restores detection via the rules watch event alone, with no agent restart.
Four fixes to the reloader introduced with the no-restart policy reload, plus the coverage the original report asked for: - The reloader now runs whenever a trust-policy path is configured. A boot policy that fails verification no longer disables reloading: signing stays off, the agent says so, and the first verifiable mounted artifact enables it without a restart. Refusing to start was rejected: a crash-loop on a bad policy hands anyone with ConfigMap write access a fleet-wide kill switch. - The policy pointer read by reconciler ticks and written by the reload goroutine is now atomic; pinned by a -race test. - Reload-apply resyncs the rules watcher immediately (ResyncNow), so a stricter policy drops already-loaded rules within one reload interval instead of waiting for the next Rules watch event. Rule bindings still re-evaluate on their next binding event; logged as such on reload. - Rollback replay refused: policyVersion is a monotonic counter inside the signed policy content, mirroring the fragment version guard. Hash dedup only detects change; it is not a rollback defense. Absent version means 0, so existing artifacts keep verifying and reloading unchanged. - The permissive direction is no longer silent: a policy without ruleClasses logs 'rule signing DISABLED' at boot and on every applied reload, and a reload that narrows scope (rule signing off, binding signing off, enforce to alert) logs 'trust policy reloaded with REDUCED scope'. Tests: cross-class trusted key on Rules fragments (a base-trusted key signing an overlay is refused with ErrSignerNotTrusted — the reported case), reloader recovery from an invalid boot artifact, version rollback refusal with unversioned back-compat, resync flips admission both directions without watch events, concurrent policy swap under -race, and a cross-class-trusted-key archetype in the sync differential (30 cells).
After a refused reload the mounted ConfigMap contains the REFUSED artifact, so the cluster is the wrong place to ask what is enforced. The agent now answers: - Reload events carry two digests: the candidate artifact's and the in-force policy's (plain sha256 hex, matching sha256sum on the mounted file). The dedup hash and the in-force hash are separate — a refused artifact updates only the former. - The refusal log names both digests, the root fingerprint, and states that the cluster now shows an artifact that is not enforced; the apply and boot logs name the in-force digest. - /policyz on the existing health port serves the in-force identity: digest, mode, root anchor (compiled|mounted|demo), policy version, class counts, last refused digest, and the rules watcher's admitted/rejected/ effective counters (the queryable form of the detection-outage backstop). Digests, mode and counts only — never the policy body, never signer lists: an unauthenticated port must not bypass ConfigMap RBAC or enumerate trusted signers. 404 until a policy was ever in force. - Observability is fail-open: status bookkeeping never blocks an apply, and killing the health server does not affect enforcement. Tests: digest presence on refusal and apply events, /policyz leak-boundary body assertions and 404 semantics, status lifecycle (counters survive applies, refusal history survives, snapshots are copies, counters alone never fabricate an in-force policy).
…gned content Editing a signed fragment's stored spec is inert by design — enforcement binds the embedded signed content — but nothing said so: kubectl showed injected content on an object that still reported a valid signature, and the reverse read equally wrong (an operator adding an exec by patching assumes it is in force). The divergence is now reported, in both directions, on every surface that reads signed objects: - DiffSpecPaths over an explicit spec-path table; a test pins the table to setSpecPaths' field list so a new spec field cannot join only one of the class-confinement check and the divergence diff. nil and empty collections compare equal at every level (the embed canonicalization materializes empty maps where stored objects have nil). - Bundle fragments (assembly), flat signed profiles (cache verify), and admitted Rules objects (watcher sync) each warn: 'stored spec is display-only and is NOT enforced; enforcement uses the signed content' — path names and counts only, never values, so nothing an attacker writes into a stored spec reaches a log pipeline. One warning per distinct stored content, re-armed when the divergence clears or the content changes again. - Observability only: divergence never enters admission, and a comparison failure reports nothing. In enforce mode a distinct low-severity drift alert (R1017 'Signed profile drift') is emitted — deliberately not R1016, which keeps meaning 'enforced content tampered'. Tests: table/confinement consistency, diff semantics, stored-edit leaves composite and Merkle root byte-identical while divergence reports, dedup and re-arm across repeated syncs, values never logged, alert-mode emits no alert while enforce emits R1017, Rules analogue loads only signed rules.
…isible The three outcomes of resolving kubescape.io/user-defined-profile under a trust policy were implemented and test-pinned but silent from the operator's side. Now each states itself: - Outcome 2 (verifying bundle shadows the same-named classical profile): 'signed bundle overlay shadows a ContainerProfile of the same name: the bundle is enforced, the named profile is not' — Info, once per root transition (rides the existing transition gate, so steady state costs no storage reads); a same-named object that is itself a bundle member is not shadowed. - Outcome 3 (unverifiable fragments, no prior projection): the pending warning now states the consequence — NO fallback to the same-named profile, container runs with no user-defined profile until the bundle verifies — instead of the generic legacy-AP/NN wording, and is counted by a new user_defined_bundle_unverifiable metric (interface + all four managers). Fallback stays rejected: it would let anyone able to corrupt one fragment downgrade a signed bundle to an unsigned profile. Tests: exactly one shadow log per root transition and none on unchanged re-assembly, no shadow log without a same-named profile, outcome-3 wording present and legacy wording absent, the three existing shadowing tests unchanged.
…gence Test_46: invalid boot policy screams and stays recoverable; a valid policy mounted later enables signing with zero restarts; scoping up to ruleClasses drops the unsigned baseline via resync with no Rules watch event; a refused reload names both artifact digests, and sha256sum of the mounted files matches them. Restart counts asserted unchanged across every post-boot phase. Test_47: editing the stored spec of an embedded-content fragment reports the display-only divergence without R1016 and without spec values in the log, and an exec outside the SIGNED content still raises R0001 — the composite is unchanged.
Test_40 grepped the pre-reloader boot-failure wording; the line is now 'trust policy invalid at startup: signed bundle overlays DISABLED …'. Test_46 waited on markers that repeat across its own phases: 'trust policy reloaded without restart' and 'reload REFUSED' both appear in earlier phases, so the waits passed against stale lines and the real assertions ran before ConfigMap propagation. Every wait now uses a phase-unique marker (the refused artifact's own sha256 for the refusal phase) with windows sized for mount propagation plus the reload poll.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multi-party signed profile fragments assembled into one enforced ContainerProfile.
pkg/signatureported to the ContainerProfile world (single CP adapter; AP/NN adapters removed — the fork storage dropped those types), R1016 tamper detection on user-authored CPs (verify on load + on reconciler refresh).pkg/signature/bundle: per-class trust policy (allowed signer fingerprints + allowed spec paths), deterministic order-independent assembly, Merkle leaf-tree manifest, internal re-sign of the composite with the cluster key.signature.kubescape.io/bundlelabel (membership re-checked client-side — the storage List ignores label selectors and returns spec-stripped items, so fragments are fetched individually), fail-closed on any inadmissible/tampered fragment, composite carries its Merkle root as ResourceVersion for the refresh fast-skip.bundleTrustPolicyPath+bundleSigningKeyPath(+enableSignatureVerificationfor strict mode).cmd/sign-objectported to ContainerProfile; binaries released assign-object-v0.1.0.Validated: component tests green (31/31, run 31244214794 + rename-confirm run) and a full live demo dry-run against a k3s cluster (bob PR kubescape#197).