feat(core): set-operation leg schemas decided before any row; a denied GET / falls back to the build major

[fupelaqu]

Summary

Set-operation legs get their column schemas before any row is read, so a consumer can type each leg up front instead of from its first rows.

• SetOperationValues.legSchemas gives, per leg and column, the type at every depth (struct fields, list elements) and a reader that answers that type's class.
• Dates are routed from their mapping format, never from a value.
  • A format core cannot read for every value becomes VARCHAR, the text as stored, for every row. That covers zone names, region ids, unmeasured letters, and Elasticsearch 6.8's AM/PM with no hour.
  • Every other date keeps its type and is read exactly as legReaders reads it.
• No slot (values typed as they come, as today) where:
  • the leg's or the column's type is not known at some depth (ANY, a struct with no known fields);
  • the leg is NULL or SELECT *;
  • the leg is NUMERIC, TEMPORAL, VARBINARY or REAL.
• VersionApi.dateDialectVersion: the cluster's version, else the major the client module is built for, for a user whose role is denied GET /.
  • It is asked once per client and forwarded by ElasticClientDelegator.
  • Core's own UNION ALL reads dates through it too: one code line in SearchApi.setOperationVersion.
• Unchanged: the existing readers (converter, readAs, dateByFormat, positionConverter, legReader, legReaders, converters). The diff adds only, apart from that one line and three comments.

Behaviour changes (release notes)

• A user denied GET /:
  • Core's UNION ALL now reads dates as the client's build major. Before, a converting date failed ("… which is not known"). Example: SELECT d AS k FROM t UNION ALL SELECT c_c1 AS k FROM t, with c_c1 formatted yyyy/MM/dd, now answers 2024-01-31T00:00Z.
  • Agreeing custom-format dates are read as for a privileged user.
  • The version is asked once per client. A privilege granted later is seen after a client restart.
• Downstream rebuild: VersionApi gains a member (a lazy val). Projects that implement VersionApi must be rebuilt.

Evidence

Check	Result
New specs: DateRouteSpec, SetOperationLegSchemasSpec, DateDialectVersionSpec, LegSchemasConsumerSmokeSpec	green
Every format of the es6 / es7 / es8 date corpora routed on its own major	6,323 / 4,721 / 4,300 formats. 866 / 487 / 461 are VARCHAR as stored, exactly the formats with an unreadable row.
legReaders / converters / columnTree outputs, main vs branch	3,472 statements, 41,586 lines, byte-identical
A slot's reader output matches its declared type at every depth	population test over the spec's schema pairs
GET / asked once per client through MetricsElasticClient	10 + 10 statements → 1 call, 1 ERROR line
core/test (clone of the commit)	1,244 / 1,244
+ compile (2.12 + 2.13) and lint (headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck) on the commit; the 5 client modules (es9 on JDK 17) and checkLog4j on the first implementation (no dependency changed since)	green
ES 8.18.3: SetOperationTypeSpec + UnionAllCompletenessSpec	24 / 24
ES 6.8.23 REST: SetOperationTypeSpec + UnionAllCompletenessSpec	24 / 24

Performance

main 7094067 vs this branch, runs interleaved (main, branch, main, branch).

• Each run warms the JIT first and reports medians.
• Setup is timed apart from the row loop.
• 100,000 rows per leg.
• Statements:
  • converting: d ∪ c_c1 + n ∪ x;
  • agreeing: ts ∪ ts;
  • zone-name: tsz ∪ ts.

Median	main (run 1, run 2)	branch (run 1, run 2)
UNION ALL row loop, ns/row: converting	193, 194	196, 203
… agreeing	12, 11	15, 11
… zone-name	540, 528	513, 487
legReaders read, ns/value: converting / agreeing / zone-name	84 / 44 / 532, 81 / 43 / 468	78 / 45 / 495, 81 / 44 / 485
unionAllRowMappers build, µs: converting / agreeing / zone-name	248 / 68 / 123 (cold JVM), 34 / 9 / 22	26 / 11 / 25, 34 / 10 / 21
legSchemas build, µs (new; only its callers pay it)	—	39 / 15 / 23, 46 / 16 / 26
legSchemas slot read, ns/value	—	84 / 44 / 26, 85 / 43 / 25

• Existing paths are unchanged within noise. Their code is byte-identical; only setOperationVersion changed.
• The new plan-time cost is 15–46 µs per statement, paid only by callers of legSchemas. Parse and resolve cost 130–420 µs for the same statements.
• A zone-name date read as stored costs about 25 ns per value, against about 490 ns when parsing is attempted.

Not in this PR

• The Arrow Flight SQL server consumes legSchemas in a follow-up change, after this one is released.
• Pre-existing, not widened: legReaders fails every object at a struct field typed ANY. legSchemas gives no slot there.

🤖 Generated with Claude Code