From bfccedc93e54dbabe0693de152f85ad7aa49e5e1 Mon Sep 17 00:00:00 2001 From: Said Boudjelda Date: Tue, 20 Oct 2026 15:11:17 +0200 Subject: [PATCH] docs(plans): the conformity spec and its verified review, with what shipped Two planning documents that stayed untracked in the main checkout: - plans/conformity-spec.md: the design of KafkaReference and POST /api/conformity/check. A "Shipped" row now names the PRs that delivered it (#407 to #416, then #419 to #433), and its file references point at cli/examples/conformity, where the schema and examples live. Its inline examples stay as the 2026-10-19 drafts and say so. - plans/conformity-enhancements.md: the four-lens review of 2026-10-20, 29 verified findings. An Outcome table maps each finding to the PR that fixed it: 17 fixed, 2 partly covered by the book chapter, 10 open. The draft schema and examples in plans/conformity/ are left out: the shipped copies in cli/examples/conformity supersede them, and the draft schema lacks kafka.source and topics. --- plans/conformity-enhancements.md | 292 +++++++++++++ plans/conformity-spec.md | 708 +++++++++++++++++++++++++++++++ 2 files changed, 1000 insertions(+) create mode 100644 plans/conformity-enhancements.md create mode 100644 plans/conformity-spec.md diff --git a/plans/conformity-enhancements.md b/plans/conformity-enhancements.md new file mode 100644 index 00000000..9878ce16 --- /dev/null +++ b/plans/conformity-enhancements.md @@ -0,0 +1,292 @@ +# Conformity: verified enhancement findings (2026-10-20) + +Review of the conformity feature as merged on main 15688d1 (PRs #407-#416): four read-only reviewers (correctness, coverage, usability, robustness), each finding then checked by a skeptical verifier. All 29 below were confirmed; 0 were refuted. Line references are to `15688d1`. + +## Outcome (2026-10-20) + +Seventeen findings are fixed on `main`, two are partly covered, and ten are open. Each row names the PR that closed the finding: + +| Finding | Outcome | +| --- | --- | +| `cli-yaml-drops-duplicates`, `yaml-conversion-drops-content`, `cli-yaml-alias-recursion` | Fixed in #421: a repeated key, a second document and a merge key are refused, and aliases are bounded | +| `brokers-source-wrong-cluster`, `admin-cluster-identity`, `brokers-partial-answer`, `brokers-null-config-value` | Fixed in #422: the brokers source checks the cluster id and reads every broker the pools expect | +| `require-listener-renders-go-map` | Fixed in #423 | +| `empty-subjects-pass`, `zero-replica-pools-counted`, `podmonitor-namespaceselector`, `junit-warn-unknown-error`, `subjects-cap-silent` | Fixed in #424 | +| `per-topic-durability` | Fixed in #419: `spec.topics` | +| `refusal-one-line-one-error` | Fixed in #420: `check` lints first, and lint lists every finding with its line and column | +| `book-describes-v1-only` | Fixed in #431: the Conformity Checks chapter covers every section | +| `table-too-wide` | Fixed in #433: details go on lines under each row | +| `no-rule-catalogue-for-authors` | Partly, in #431: the chapter has the language at a glance; the schema still has few descriptions | +| `book-source-brokers-access` | Partly, in #431: the chapter says `kafka.source: brokers` needs the Kates API's Kafka connection, not which ACLs | +| `rolling-restart-silent-header`, `external-listener-scope`, `cr-operands-and-guards`, `role-scheduling`, `storage-class-properties`, `junit-not-ci-ready`, `snapshot-first-reference`, `mcp-conformity-check`, `no-fix-pointer`, `nodes-full-list` | Open | + +The findings below are as the review wrote them. + +## Correctness + +### The CLI silently drops duplicate YAML keys and any second document, so rules vanish from the check + +`cli-yaml-drops-duplicates` · medium · bug · size XS + +- **Evidence:** cli/cmd/conformity.go:82-91 (yaml.Unmarshal into a yaml.Node reads only the first document), cli/cmd/conformity.go:99-108 (m[n.Content[i].Value] = v, so the last duplicate wins); this defeats kates/src/main/java/com/bmscomp/kates/conformity/KafkaReferenceReader.java:53-57,113 (STRICT_DUPLICATE_DETECTION, added so that 'a typo would [not] drop a rule without a word'). I checked yaml.v3 v3.0.1 in a scratch module: decoding into a yaml.Node returns nil on a duplicate key and on a second '---' document. +- **Scenario:** A reference has `spec.brokers: {count: {min: 3}, storage: {...}, zones: {min: 3}}`, and later in the file a second `brokers:` block with only `antiAffinity: required` (a copy-paste or a merge of two files). The CLI sends only the second block, so brokers.count, the storage rules and brokers.zones are never evaluated. The table hides PASS rows, so the drop goes unnoticed, and a cluster that later loses a broker still exits 0 in CI. The same happens to a repeated `kafka.config` key, and to a file whose rules continue after a `---`. +- **Proposal:** In conformityValue's MappingNode case, keep a map of key to line and return `line N: is already set at line M` on a repeat. In conformityJSON, decode with yaml.NewDecoder and refuse the file when a second Decode does not return io.EOF ('holds more than one document'). Add one test for each to conformity_test.go. +- **Verifier:** Confirmed. cli/cmd/conformity.go:83-84 calls yaml.Unmarshal into a yaml.Node, and :106 does m[key]=v, so a repeated key is overwritten silently. I copied conformityJSON/conformityValue into a scratch module pinned to yaml.v3 v3.0.1, the version in cli/go.mod:24. For `spec.brokers{count,zones}` followed by `brokers{antiAffinity}` it printed {"spec":{"brokers":{"antiAffinity":"required"}}} with no error. A second `---` document was dropped without an error. A JSON file with a duplicate key behaved the same way. The API only ever gets the merged JSON map, so KafkaReferenceReader's STRICT_DUPLICATE_DETECTION (KafkaReferenceReader.java:113, intent comment at :53-58) never sees the duplicate. No CLI test covers either case (cli/cmd/conformity_test.go has no duplicate or multi-document test). Severity medium: rules leave a CI gate without any message. + +### A rule with no subjects passes: storage.class passes on ephemeral storage + +`empty-subjects-pass` · low · bug · size XS + +- **Evidence:** kates/src/main/java/com/bmscomp/kates/conformity/ConformityCheck.java:366-376 (with no PVCs, classes is empty and the detail says 'no PVCs: ephemeral storage'), ConformityCheck.java:719-726 (an empty values map makes `failing` empty, so the status is PASS). The same empty-map PASS happens in perPod (675-681), perNode (404-412), storage.type (350-358), perListener (970-975) and effectiveConfig with no brokers (550-562). podMonitor's allMatch over zero broker pods also passes vacuously (483-489). Spec §4.3 says storage.class is checked per PVC. +- **Scenario:** A reference declares `brokers.storage.class: gp3` (with no size rule) against a cluster whose broker pool uses `type: ephemeral`. No PVC exists, so the rule is PASS with observed null. The CLI hides PASS rows, so the 'ephemeral storage' detail is never shown and the check exits 0. The same goes for a JBOD pool whose volumes are all ephemeral, and for any per-pod rule on a role that currently has no pods. +- **Proposal:** In rule(), when observation.values() is empty, return FAIL (WARN for a warn rule) instead of PASS. Use the observation's detail, or 'nothing to check' when it has none. In the podMonitor rule, require !brokerPods.isEmpty() before allMatch. Add a test with an ephemeral pool that asserts storage.class is FAIL with the ephemeral detail. +- **Verifier:** The mechanism is confirmed. In ConformityCheck.java:366-376, with no PVCs the classes map is empty, and rule() at :719-726 then has nothing failing, so the result is PASS. The CLI hides PASS rows that have no Note (conformity.go:141), so the 'no PVCs: ephemeral storage' detail is never shown. No test covers the ephemeral case (ConformityCheckTest:351-366 only tests bound PVCs). Downgraded to low: on ephemeral storage, storage.size observes 0 through boundCapacity (:927-947, spec §4.3 'an ephemeral pool as 0') and storage.type observes 'ephemeral'. The shipped prod-3az.yaml:44-47 declares class together with type and size, so it already fails there. The false PASS needs a reference that declares only class, or a role with no pods at all. The blanket 'empty means FAIL' proposal needs care: perNode with no pods would also flip, and that one is a correct vacuous PASS. + +### With kafka.source: brokers, the check reads whichever cluster the Admin client reaches, never checked against the target Kafka CR + +`brokers-source-wrong-cluster` · low · bug · size XS + +- **Evidence:** ConformityCheck.java:168-180 (describeCluster and describeConfigs go through admin.sharedAdminClient()). KafkaAdminService.java:38,68 bootstraps from kates.kafka.bootstrap-servers, set by charts/kates/templates/configmap.yaml:8 and by `kates deploy` (cli/cmd/deploy_components.go:1701). The target is resolved from kates.topology.kafka-namespace/kafka-cluster (ConformityCheck.java:101-102, 208-238), which the chart never sets. Only the CR's status.clusterId is compared with target.clusterId (229-237). The report's target.clusterId comes from the CR (266). +- **Scenario:** Kates' bootstrap points at another cluster (for example charts/legacy-kafka, or a second Strimzi cluster), while kafka/krafter also exists. A reference with `kafka.source: brokers` and `config: {min.insync.replicas: {min: 2}}` is checked against the other cluster's brokers. The report still names krafter and its cluster id, so it shows PASS or FAIL for a cluster it never looked at. Pinning target.clusterId does not help, because that pin is compared only with the CR. +- **Proposal:** In the brokers Source, also await describeCluster().clusterId(). When the Kafka CR has a status.clusterId and the two differ, throw (so the Source turns it into Unreadable) with 'the Kafka Admin client (kates.kafka.bootstrap-servers) reaches cluster X, not Y'. Every brokers-mode rule is then UNKNOWN, never a wrong PASS or FAIL. Mock clusterId() in the brokers() test fixture and add one mismatch test. +- **Verifier:** The gap is confirmed. In brokers mode, ConformityCheck.java:168-180 reads through admin.sharedAdminClient(), which bootstraps from kates.kafka.bootstrap-servers (KafkaAdminService.java:36-41,68). The target comes from kates.topology.* (ConformityCheck.java:101-102). The chart's configmap.yaml:8 sets only the bootstrap, and `kates deploy --kafka-name X` (deploy.go:117, deploy_components.go:1676,1701) points the bootstrap at X without setting the topology. The clusterId pin is compared only with the CR (:229-237), and the report takes the clusterId from the CR (:266). Nothing compares describeCluster().clusterId(). Low: a wrong report needs a bootstrap/topology mismatch and also a Kafka CR kafka/krafter that is a different cluster. With only the mismatch, target() returns 409 'No Kafka kafka/krafter' (:226-227), which is safe. The guard costs about 5 lines. + +### During a rolling restart, count and zones can FAIL while the report header still says the cluster is settled + +`rolling-restart-silent-header` · low · ux · size XS + +- **Evidence:** ConformityCheck.java:126-137: the pods Source drops pods that have a deletionTimestamp, and clusterState (772-785) counts podsNotReady from that same filtered list. Strimzi keeps the Kafka CR Ready and generation == observedGeneration during a manual or certificate-renewal roll. Spec §5.4 promises a header that says results may change while the cluster settles. +- **Scenario:** A 3-broker cluster is mid-roll (strimzi.io/manual-rolling-update, or a CA renewal). broker-4 is terminating or not yet recreated. brokers.count {min: 3} is FAIL (observed 2), and brokers.zones {min: 3} is FAIL if broker-4 was alone in its zone. The header says 'Ready' and shows no not-Ready pods, so a CI gate fails with no hint that a roll is in progress. +- **Proposal:** Keep terminating pods in the pods Source. Filter them out in rolePods (so D1 is unchanged), and count them in clusterState as not Ready, or as a separate `podsTerminating` that conformityClusterLine prints. Optionally add 'N terminating' to the count rule's detail. +- **Verifier:** Confirmed. The pods Source drops pods with a deletionTimestamp (ConformityCheck.java:136). clusterState counts podsNotReady from that same filtered list (:775), so a terminating broker is neither counted nor reported as not Ready. The count rule's detail at :297 ('2 of 2 Ready') gives no hint either. A roll triggered by an annotation or a CA renewal changes neither the Kafka CR's generation nor its Ready condition, so conformityClusterLine (conformity.go:164-180) prints 'Ready'. Spec D1 (plans/conformity-spec.md:655) says counts should not 'turn into a FAIL during a rolling restart'. Excluding terminating pods breaks that for the window between a pod's deletion and its replacement being created. So the case for counting terminating pods is stronger than for a header hint alone. The window is transient (about the grace period per broker), hence low. + +### Node pools scaled to 0 replicas still decide storage.type and controllers.dedicated + +`zero-replica-pools-counted` · low · bug · size XS + +- **Evidence:** ConformityCheck.java:350-358 (storage.type takes every pool whose spec.roles has the role) and 301-311 (dedicated flags every pool that has both roles). Neither reads spec.replicas or status.nodeIds. +- **Scenario:** A team follows Strimzi's procedure for changing storage type: create a jbod broker pool, move the partitions, and scale the old persistent-claim pool to `replicas: 0`. While that empty pool exists, `brokers.storage.type: jbod` FAILs with subject old-pool, even though no broker runs on it. An empty combined pool left at 0 replicas likewise fails `controllers.dedicated: true`. +- **Proposal:** Skip pools whose spec.replicas is 0 (or whose status.nodeIds is empty) in both rules, and name them in the detail ('old-pool: 0 replicas, not counted'). Add one test pool with replicas 0. +- **Verifier:** The behavior is confirmed. storage.type (ConformityCheck.java:350-358) and controllers.dedicated (:301-311) look only at spec.roles and never at spec.replicas or status.nodeIds, so an empty pool still decides both rules. This is borderline: spec §4.3 defines both rules per declared pool ('Per pool. Subjects: pools'; 'No pool of this cluster has both roles'), so the FAIL matches the spec as written. The state is also transient: Strimzi's storage-change and role-migration procedures delete the old pool. It is real but of little value; at most, name the 0-replica pool in the detail. + +### A PodMonitor with namespaceSelector {any: false} or {matchNames: []} is read as watching no namespace + +`podmonitor-namespaceselector` · low · bug · size XS + +- **Evidence:** ConformityCheck.java:1008-1015: a non-empty selector map counts only when any is true or matchNames names the namespace. prometheus-operator (getNamespacesFromNamespaceSelector) treats any=false with no matchNames as the PodMonitor's own namespace. +- **Scenario:** A PodMonitor in namespace kafka, rendered by a chart as `namespaceSelector: {any: false}` (or `matchNames: []`), with a selector that matches every broker pod. Prometheus scrapes the brokers, but monitoring.podMonitor is FAIL with 'no PodMonitor selects every broker pod'. +- **Proposal:** In watches(), return true when any is true. Otherwise, when matchNames is missing or empty, fall back to the PodMonitor's own namespace; otherwise check matchNames.contains(namespace). Extend the podMonitor test with {any: false}. +- **Verifier:** Confirmed. watches() (ConformityCheck.java:1008-1015) falls back to the PodMonitor's own namespace only when namespaceSelector is missing or an empty map. With {any: false} or {matchNames: []}, the map is non-empty, any is not true and matchNames does not contain the namespace, so it returns false. prometheus-operator's getNamespacesFromNamespaceSelector returns the monitor's own namespace when Any is false and MatchNames is empty, so the PodMonitor is scraped but the rule FAILs with 'no PodMonitor selects every broker pod'. Spec §4.5 has the same narrower wording, so the spec needs the fix too. The test helper ConformityCheckTest:662-665 covers only null and {any: true}. + +### JUnit marks an UNKNOWN warn rule as an while the check exits 0 + +`junit-warn-unknown-error` · low · bug · size XS + +- **Evidence:** cli/cmd/conformity.go:276-278 turns every UNKNOWN into tc.Error whatever its severity. ConformityCheck.java:806 counts an UNKNOWN only when its severity is fail, so the verdict is PASS. The conformity check command's help text (conformity.go:35-37) and docs/book/10-cli-reference.md:621 say 'A warn rule never fails it'. +- **Scenario:** A reference has `monitoring.podMonitor: {equals: true, severity: warn}`, and Kates' service account gets 403 listing podmonitors. The rule is UNKNOWN (warn), the verdict is PASS and the exit code is 0. But conformity.xml has an test case, and Jenkins' junit step or a GitHub JUnit reporter marks the build unstable or failed. The CI result then depends on which signal the pipeline reads. +- **Proposal:** In writeConformityJUnit, handle UNKNOWN with severity warn the way WARN is handled: a pass with the detail in system-out, prefixed 'UNKNOWN (warn)'. Keep for fail-severity UNKNOWN. Update the JUnit table in the spec §6.3 and the sentence in the book's CLI reference. +- **Verifier:** Confirmed. writeConformityJUnit turns every UNKNOWN into tc.Error without looking at severity (cli/cmd/conformity.go:276-278). The verdict counts UNKNOWN only at fail severity (ConformityCheck.java:806), and rule() gives an UNKNOWN the constraint's severity (:706-716). So a warn rule that cannot be read gives exit 0 and a JUnit . This matches the spec's JUnit table (plans/conformity-spec.md, the §6.3 table) and the book (docs/book/10-cli-reference.md:620, not :621), which map UNKNOWN to an error. It contradicts 'A warn rule never fails it' in the same paragraph and in the command help (conformity.go:37). It is a design inconsistency rather than a code slip: a small fix, low severity. + +### With kafka.source: brokers, a config that Kafka reports as null (log.retention.ms) is FAIL 'unset', though the effective value comes from a synonym + +`brokers-null-config-value` · low · bug · size S + +- **Evidence:** ConformityCheck.java:550-562: an entry that exists but has a null value is put as null (559), and `reported` is true, so the rule is FAIL with observed 'unset' and no detail. Spec §4.4 says brokers mode compares 'Kafka's default'. For log.retention.ms, log.retention.minutes, log.roll.ms and log.roll.jitter.ms, Kafka reports value=null with no synonyms and derives the effective value from the *.hours key. +- **Scenario:** A cluster keeps Kafka's default of log.retention.hours=168. A reference with `kafka: {source: brokers, config: {log.retention.ms: {min: 604800000}}}` gets FAIL on every broker with observed 'unset', although retention is exactly 7 days. +- **Proposal:** In effectiveConfig, when a broker reports a null value, either derive it for the known Kafka time families (ms <- minutes x 60000 <- hours x 3600000, read from the same Config) or throw Unreadable('brokers report no value for X; Kafka derives it from log.retention.hours'). Then the rule is UNKNOWN, not a false FAIL. Add one test with a null-valued entry. +- **Verifier:** Confirmed. In effectiveConfig (ConformityCheck.java:550-562), an entry that is present with a null value sets reported=true and stores null. rule() at :720-721 counts a null as failing, so the result is FAIL with observed 'unset' (render :843-846) and no detail. Kafka 4.2's ServerLogConfigs in ~/.m2, read with javap, documents LOG_ROLL_TIME_MILLIS and LOG_ROLL_TIME_JITTER_MILLIS as 'If not set, the value in log.roll(.jitter).hours is used'. log.retention.ms and log.retention.minutes have null defaults in the same way, and describeConfigs returns them as value=null. So a cluster with retention at 168 h gets a false FAIL on log.retention.ms. The XS variant (throw Unreadable, or add a detail naming the hours synonym) is enough; deriving the value is S. Low: the reference author can write log.retention.hours instead. + +## Coverage + +### The reference can check broker defaults for replication factor and min.insync.replicas, but not the values each topic actually has + +`per-topic-durability` · high · gap · size M + +- **Evidence:** kates/src/main/java/com/bmscomp/kates/conformity/KafkaReference.java:80-85 (kafka.config is the only durability setting); ConformityCheck.java:172-180 (describeConfigs reads BROKER resources only), :529-538 (with source cr it reads spec.kafka.config only); kates/src/main/java/com/bmscomp/kates/service/SecurityService.java:585-604 (the audit already lists and describes every topic, but flags only RF<2, as WARN, and never per-topic min.insync.replicas), :1715,1735-1739 (the secret scan already runs one describeConfigs over every TOPIC resource); /Users/bmscomp/codes/kates/plans/conformity-spec.md:604 (cut because of scale) +- **Scenario:** prod-3az declares kafka.config default.replication.factor {min: 3} and min.insync.replicas {min: 2}, and both PASS. A team creates `payments` with --replication-factor 2 --config min.insync.replicas=1 (or a KafkaTopic with replicas: 2). With one broker down, an acks=all write is accepted by a single replica, and a second loss loses data. The conformity verdict is PASS. `kates security audit` also passes this topic, because it only flags RF=1 and never reads topic-level min.insync.replicas. +- **Proposal:** Add an optional `topics` section: {replicationFactor: Constraint, minInsyncReplicas: Constraint, exclude: [name prefixes, at most 50]}. Read it through the pooled Admin client, and only when the section is declared: listTopics (internal topics included), describeTopics, and one describeConfigs over the TOPIC resources. These are the calls the audit and the secret scan already make. A topic's RF is its smallest replica set across partitions, and minISR is the topic's effective value. Subjects are topic names, capped by the existing MAX_SUBJECTS; the detail reads 'N of M topics'. If Admin is unreachable the rule is UNKNOWN. Use prefixes, not regexes, so spec section 8's 'no regexes' still holds. Do not read KafkaTopic CRs: the unidirectional topic operator creates none for topics that clients create. The book should say that declaring `topics` opens a Kafka connection even with source cr. The existing mock-Admin tests (ConformityCheckTest.java:536-590) cover the test setup. +- **Verifier:** Confirmed on 15688d1. KafkaReference.java:80-85: the Kafka section is only source, rackAwareness, authorization, listeners and config, with no topic-level field. ConformityCheck.java:172-180: the Admin path describes BROKER resources only. :529-538: source cr reads only spec.kafka.config. So a topic created with RF 2 and min.insync.replicas=1 never reaches any rule, and prod-3az's kafka.config rules (prod-3az.yaml:67-68) still PASS. Nothing else in the backend gates on this either. SecurityService.java:590-603 flags only partitions with replicas().size() < 2, as WARN, and skips '__' topics, so RF 2 passes. A grep of kates/src/main/java shows no per-topic min.insync.replicas check anywhere: SecurityService.java:220 reads the broker value, and :1735-1739 describes TOPIC configs only to scan them for secrets. The failure mode is right: with RF 2 and minISR 1, one broker down still accepts acks=all writes on a single replica. conformity-spec.md:604 cut this on purpose for scale. The proposed fixes for that (an opt-in section, prefix excludes, the existing MAX_SUBJECTS=50 cap at ConformityCheck.java:84, UNKNOWN through Source when Admin is down) are workable. The mock-Admin test setup it would reuse is at ConformityCheckTest.java:533-563. Size M holds: model, reader, schema, check, tests and book. + +### Listener rules cannot target only exposed listeners, so 'TLS outside the cluster' needs a dated waiver that also hides new gaps + +`external-listener-scope` · medium · gap · size S + +- **Evidence:** kates/src/main/java/com/bmscomp/kates/conformity/KafkaReference.java:91,115 (require matches by name, every applies to all listeners); ConformityCheck.java:626-646 (every evaluates every listener); cli/examples/conformity/prod-3az.yaml:63-64,81-83 (every.tls needs a dated waiver for the internal plain listener on 9092); charts/kafka-cluster/values.yaml:93-113 and charts/kafka-cluster/values-prod.yaml:67-69 (the chart's own production shape is internal plain SCRAM plus external TLS) +- **Scenario:** Policy: exposed listeners use TLS, and in-cluster plain SCRAM is allowed. A cluster built from values-prod.yaml has plain (internal, tls false), tls, and external (nodeport, tls true). `every.tls: true` FAILs on plain for good, so the team waives kafka.listeners.every.tls until a date and renews it every quarter. Later someone adds `partners` (loadbalancer, tls false, scram-sha-512), which sends SCRAM credentials in plaintext over the internet. require[external] still passes and every.auth passes. every.tls does fail on partners, but the waiver covers the whole rule, so the result shows WAIVED and the verdict is PASS. +- **Proposal:** Add `kafka.listeners.external: {tls, auth}`, with the same ListenerProperties shape minus type. It is evaluated over listeners whose type is route, loadbalancer, nodeport or ingress. Rule ids are kafka.listeners.external.tls and .auth, with the exposed listener names as subjects. With no exposed listener the rule PASSes, with the detail 'no exposed listener'. This is about 15 lines in listeners(), reusing perListener. The reader and schema reuse the existing listener-property validation. +- **Verifier:** Confirmed. ConformityCheck.java:626-646: every.type, .tls and .auth run perListener over every listener in the CR, with no way to scope them. Waivers apply to a whole rule: waive() at ConformityCheck.java:751-766 keys on rule id only, and the reader at KafkaReferenceReader.java:414-418 accepts only a rule id, never a subject. So once kafka.listeners.every.tls is waived for the internal 9092 plain listener, as prod-3az.yaml:81-85 does, a new non-TLS loadbalancer listener is also WAIVED and the verdict stays PASS. Today the only way to target exposed listeners is require by name, which cannot catch a listener added later. The chart's own shape matches the scenario: values.yaml:93-105 has internal plain with SCRAM, and values-prod.yaml:67-69 adds a nodeport listener with TLS. Size S is fair. An alternative to weigh is a waiver scoped to subjects (for example subjects: [plain]). That fixes the hiding for every per-subject rule, including the proposed topics rule. + +### The reference cannot check Kafka CR settings the chart's prod profile depends on: Cruise Control, the entity operator, the disk quota and the PDB + +`cr-operands-and-guards` · low · gap · size S + +- **Evidence:** kates/src/main/java/com/bmscomp/kates/conformity/KafkaReference.java:29-37,80-85,118-119 (no field for spec.cruiseControl, spec.entityOperator, spec.kafka.quotas or spec.kafka.template.podDisruptionBudget); ConformityCheck.java:472-477 (monitoring.kafkaExporter is the same one-line presence read on the Kafka CR it already holds); ConformityCheck.java:542-547 (a key outside spec.kafka.config is 'not set in the Kafka CR'); charts/kafka-cluster/values.yaml:178-179 (PDB maxUnavailable in the Kafka CR template), :338-356 (Cruise Control on with autoRebalance); charts/kafka-cluster/values-prod.yaml:63-66 (quotas type strimzi, minAvailableRatioPerVolume 0.1, 'a full log directory goes offline'); charts/kafka-cluster/templates/kafka.yaml:56-58,114-118,130; /Users/bmscomp/codes/kates/plans/conformity-spec.md:606,608 (cut as separate components and for having no PDB RBAC, but Strimzi takes the PDB from the Kafka CR, which is already read) +- **Scenario:** A values change sets cruiseControl.enabled=false or drops kafka.quotas. Scale-ups stop rebalancing because the add-brokers autoRebalance is gone, and a broker whose log dir fills goes offline with no quota to stop producers first. prod-3az still PASSes, and the schema has no field to catch it. Trying `kafka.config[client.quota.callback.class]` with source cr FAILs 'not set in the Kafka CR' even when quotas are on, because Strimzi renders quotas outside spec.kafka.config. A Drain Cleaner setup needs maxUnavailable: 0; if someone resets it to 1, the cluster goes on to pass every check. +- **Proposal:** Read these from the Kafka CR already fetched: no new source, no new RBAC, about 5 lines each in the style of kafkaExporter. Add `operands: {cruiseControl, topicOperator, userOperator}` as booleans (spec.cruiseControl, spec.entityOperator.topicOperator and .userOperator present), `kafka.quotas` as an enum none|kafka|strimzi (spec.kafka.quotas.type, none when absent), and `kafka.maxUnavailable` as an integer (spec.kafka.template.podDisruptionBudget.maxUnavailable, 1 when absent, Strimzi's default). Cruise Control goals stay out: they belong to KafkaRebalance content, not to presence. +- **Verifier:** The gap is real. KafkaReference.java:29-37, 80-85 and 118-119 have no field for spec.cruiseControl, spec.entityOperator, spec.kafka.quotas or spec.kafka.template.podDisruptionBudget. declaredConfig at ConformityCheck.java:542-547 reads only spec.kafka.config, so with source cr a quota key reports 'not set in the Kafka CR'. Each read would be a one-liner on the Kafka CR already fetched, in the style of kafkaExporter at :472-477, needing no new RBAC. Strimzi does take the PDB from the Kafka CR template, as values.yaml:177-179 shows. I downgraded it because the case is weaker than stated. (1) conformity-spec.md:606 and :608 cut these on purpose. (2) Quota presence can already be checked: with kafka.source brokers, kafka.config[client.quota.callback.class] reads the effective broker config, a known Kafka key that Strimzi sets for quotas.type strimzi. (3) The chart always renders entityOperator.topicOperator and .userOperator (templates/kafka.yaml:114-118), so the entity-operator part only helps clusters not built from the chart. (4) The impact is operational (rebalance, drain behaviour, disk full), not silent data loss. The PDB maxUnavailable and Cruise Control presence checks are still cheap and worth it. Size S holds. + +### A role cannot require a priority class or dedicated nodes, although the check already reads the pods and node labels it would need + +`role-scheduling` · low · gap · size S + +- **Evidence:** kates/src/main/java/com/bmscomp/kates/conformity/KafkaReference.java:50-59 (Role has no scheduling fields); ConformityCheck.java:126-137 (full pod specs already listed), :158-165 (node labels already loaded for zones); charts/kafka-cluster/values.yaml:230-231 (the chart exposes per-pool tolerations and priorityClassName) +- **Scenario:** A platform runs brokers on a tainted `kafka` node group with priorityClass kafka-critical. A new pool brokers-az4 (for example one written by `kates detect --generate-values`) has no priorityClassName and no nodeSelector or tolerations. Its pods land on general nodes shared with batch jobs and are the first preempted or evicted under memory pressure. count, zones, skew and perNode all PASS, and the verdict is PASS. +- **Proposal:** Add `.priorityClass: Constraint`, compared per pod against spec.priorityClassName (`unset` when empty). Add `.nodeLabels: {