Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
package app.softnetwork.elastic.sql.bridge

import app.softnetwork.elastic.sql.{PainlessContext, PainlessContextType, PainlessTarget}
import app.softnetwork.elastic.sql.`type`.SQLTemporal
import app.softnetwork.elastic.sql.query.{
Asc,
BucketIncludesExcludes,
Expand Down Expand Up @@ -503,8 +502,11 @@ object ElasticAggregation {
}

var agg: Aggregation = {
// Story IDENT-4 -- ONE derivation of "this level is a date histogram", shared with the
// HAVING classifier (`SingleSearch.dateHistogramLevelOf`), which refuses a key filter
// this level could not carry.
bucket.out match {
case _: SQLTemporal =>
case _ if bucket.isDateHistogram =>
val functions = bucket.identifier.functions
val interval: Option[DateHistogramInterval] =
if (functions.size == 1) {
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package app.softnetwork.elastic.client

import app.softnetwork.elastic.client.result._
import app.softnetwork.elastic.sql.parser.Parser
import app.softnetwork.elastic.sql.query.SingleSearch
import app.softnetwork.elastic.sql.query.{BucketIncludesExcludes, SingleSearch}
import app.softnetwork.elastic.sql.schema.{Column, Schema, Table}
import app.softnetwork.elastic.sql.`type`.SQLTypes
import app.softnetwork.elastic.sql.PainlessContext
Expand Down Expand Up @@ -187,4 +187,54 @@ class SchemaAttachSpec extends AnyFlatSpec with Matchers {
params should include("!= null")
}
}

// -- 3. story IDENT-4: how many times each venue resolves ---------------------------------------
"a qualified GROUP BY key" should
"reach the client with its metrics on its level -- whether the FROM is resolved twice or once" in {
// `events e` is resolved AGAIN against the mapping; `events* e` is NOT -- here. Since #412 a
// client that can read a pattern's mappings resolves a pattern twice too
// (`IndicesApi.loadPatternSchema`); this double cannot (`NopeClientApi` answers no pattern), so
// it stands for every ONE-resolution venue: a remembered schema miss, a failed or denied mapping
// read, a client that is not an `IndicesApi`. The body such a venue sends is the PARSE-time one
// and must already be right. For the pattern the metric's path used to be `e.code`, the
// level's `code`.
val client = seeded()
val wrong = Seq("events e" -> true, "events* e" -> false).flatMap { case (from, again) =>
val parsed = parse(s"SELECT COUNT(*) AS c, MAX(amount) AS mx FROM $from GROUP BY e.code")
client.resolveWithSchema(parsed) match {
case ElasticSuccess(sent) =>
val level = sent.bucketTree.allTrees.flatten.head.bucketPath
val metricPaths = sent.select.fields.map(_.identifier.bucketPath).distinct
Seq(
if ((sent ne parsed) != again) Some(s"[$from] resolved again: ${sent ne parsed}")
else None,
if (metricPaths != Seq(level)) Some(s"[$from] metric paths $metricPaths, level $level")
else None
).flatten
case ElasticFailure(error) => Seq(s"[$from] ${error.message}")
}
}
withClue(wrong.mkString("\n")) { wrong shouldBe empty }
}

it should "reach the client with its key filter at both venues" in {
val client = seeded()
Seq("events e", "events* e").foreach { from =>
val parsed =
parse(
s"SELECT COUNT(*) AS c, MAX(amount) AS mx FROM $from GROUP BY e.code HAVING e.code = 'x'"
)
client.resolveWithSchema(parsed) match {
case ElasticSuccess(sent) =>
val level = sent.bucketTree.allTrees.flatten.head.bucket
withClue(s"[$from] ") {
sent.having
.flatMap(_.criteria)
.map(_.includes(level, not = false, BucketIncludesExcludes())) shouldBe
Some(BucketIncludesExcludes(values = Set("x")))
}
case ElasticFailure(error) => fail(s"[$from] ${error.message}")
}
}
}
}
26 changes: 26 additions & 0 deletions documentation/sql/dql_statements.md
Original file line number Diff line number Diff line change
Expand Up @@ -1111,6 +1111,18 @@ SELECT city, COUNT(*) AS cnt FROM dql_users GROUP BY city HAVING city <> 'Lyon';
On a `keyword`, `integer`, `long`, `double` or `boolean` key, `=`, `<>`, `IN` and `NOT IN` keep or
remove **exactly** the values written, on every supported release: a `.`, `*`, `(`, `\` or `'` in a
value is an ordinary character, and a numeric or boolean key is compared as a number or a boolean.
- The key may be written **with or without its table qualifier, on either side** — the spelling BI
tools generate: `GROUP BY c.city HAVING c.city = 'Paris'`, `GROUP BY c.city HAVING city = 'Paris'`
and `GROUP BY city HAVING c.city = 'Paris'` filter exactly the groups `GROUP BY city HAVING
city = 'Paris'` does. An unqualified name is the main table's column, as everywhere else in a
statement. In a JOIN, a column the `GROUP BY` does not group for that table (`GROUP BY a.city
HAVING b.city = 'x'`) is refused as neither a GROUP BY key nor an aggregate. A table's own name is
not a qualifier once the table is aliased (`FROM customers c … HAVING customers.city = 'x'` is
refused). A key whose function renders its own argument (`CAST`, `TRY_CAST`, `SAFE_CAST`,
`ISNULL`, `ISNOTNULL`) is matched by its spelling: write it the same way in `GROUP BY`
and `HAVING` (`GROUP BY CAST(c.city AS VARCHAR) HAVING CAST(c.city AS VARCHAR) = 'x'`); a mixed
spelling is refused, except in a JOIN, a derived table or a CTE, whose `HAVING` the relational
engine evaluates.
- ⚠️ **An integer key compared with a number that has a fractional part is refused**
(`HAVING year = 2025.5`, `HAVING year IN (2024.5, 2025)`) — whether the key is named by its
column, its SELECT alias or its GROUP BY position, or is grouped from an `UNNEST`: no integer
Expand Down Expand Up @@ -1157,6 +1169,12 @@ SELECT city, COUNT(*) AS cnt FROM dql_users GROUP BY city HAVING city <> 'Lyon';
WHOLE, and would change the counts of surviving groups whenever the key is itself a function of
the column (`GROUP BY DAY(d) HAVING YEAR(d) = 2025`). Compare the key itself, or filter in
`WHERE`.
- ⚠️ **A condition on a date grouping key is refused** when the key is a `DATE_TRUNC` key or a key
whose function returns a date
(`GROUP BY DATE_TRUNC(created, MONTH) HAVING DATE_TRUNC(created, MONTH) = '2024-01-01'`):
Elasticsearch groups such a key with a date histogram, and a date histogram has no key filter.
Restrict the rows in `WHERE` instead:
`WHERE created >= '2024-01-01' AND created < '2024-02-01' GROUP BY DATE_TRUNC(created, MONTH)`.
- A predicate naming a column that is **neither** the `GROUP BY` key **nor** an aggregate is refused
(`HAVING UPPER(name) = 'X'` when the grouping is by `city`) — with or without a `GROUP BY`.
- ⚠️ An `OR` whose branches need **different stages** is refused, because Elasticsearch applies
Expand All @@ -1167,6 +1185,14 @@ SELECT city, COUNT(*) AS cnt FROM dql_users GROUP BY city HAVING city <> 'Lyon';
`GROUP BY country, city HAVING country = 'FR' OR city = 'Paris'` would return only the groups
matching BOTH. An `OR` on ONE key, within one mechanism, is supported subject to the table
above; the corresponding `AND` is always fine, because the nesting IS the conjunction.
The same holds for two functions of one column (`GROUP BY YEAR(created), MONTH(created) HAVING
YEAR(created) = 2024 OR MONTH(created) = 1`): refused, while the `AND` of the two is accepted.

A `MATCH (city) AGAINST ('…')` on the key is a key filter like `RLIKE` and follows the same
rules: it can be combined with an aggregate by `AND`, not by `OR`; a `MATCH` over two grouping
keys (`GROUP BY country, city HAVING MATCH (country, city) AGAINST ('x')`) is an `OR` across them
and is refused; and a `MATCH` on a column that is not a `GROUP BY` key is refused, as stated
above.
- An `AND` across mechanisms is fine — each stage applies its own half.
- A group whose compared metric has no value (for instance `MAX(age)` over a group whose documents
all lack `age`) never passes a `HAVING` comparison, in either direction: the generated filter
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
package app.softnetwork.elastic.sql.bridge

import app.softnetwork.elastic.sql.{PainlessContext, PainlessContextType, PainlessTarget}
import app.softnetwork.elastic.sql.`type`.SQLTemporal
import app.softnetwork.elastic.sql.query.{
Asc,
BucketIncludesExcludes,
Expand Down Expand Up @@ -499,8 +498,11 @@ object ElasticAggregation {
}

var agg: Aggregation = {
// Story IDENT-4 -- ONE derivation of "this level is a date histogram", shared with the
// HAVING classifier (`SingleSearch.dateHistogramLevelOf`), which refuses a key filter
// this level could not carry.
bucket.out match {
case _: SQLTemporal =>
case _ if bucket.isDateHistogram =>
val functions = bucket.identifier.functions
val interval: Option[DateHistogramInterval] =
if (functions.size == 1) {
Expand Down
Loading
Loading