feat(query): a pipeable, data-last Query API next to the fluent builder

.changeset/pipeable-query-api.md

@@ -0,0 +1,5 @@
+---
+"@maple-dev/effect-orm": minor
+---
+
+A query is now `Pipeable`, and a new `Query` namespace on `/clickhouse`, `/postgres` and `/sqlite` gives a data-last form of every `CHQuery` method (`select`, `where`, `groupBy`, `having`, `orderBy`, `limit`, `offset`, `format`, `distinct`, `distinctOn`, the row locks, `route`, `crossTenant`, the table and subquery joins, `withCTE`), next to `from`, `fromQuery` and `fromUnion`: `CH.Query.from(Events).pipe(CH.Query.select(...), CH.Query.where(...))`. Each function is a `dual` that calls the fluent method, so both styles write the same SQL on every dialect, and each also has a data-first form. `$` in a data-last callback is typed from the query that `pipe` carries.

README.md

@@ -200,7 +200,7 @@ Full guides live in [`docs/`](./docs/README.md):
 | ---------------------------------------------------------- | --------------------------------------------------------------- |
 | [Getting started](./docs/getting-started.md)               | Install, define a table, build → compile → decode               |
 | [Tables and column types](./docs/tables-and-types.md)      | `table()`, column options, external tables, column types        |
-| [Building queries](./docs/queries.md)                      | `select`, `where`, `groupBy`, `orderBy`, `limit`, immutability  |
+| [Building queries](./docs/queries.md)                      | `select`, `where`, `groupBy`, `orderBy`, `limit`, immutability, `pipe` |
 | [Expressions and conditions](./docs/expressions.md)        | Comparisons, arithmetic, optional predicates, aggregates        |
 | [Joins and subqueries](./docs/joins-and-subqueries.md)     | The join family, `fromQuery`, correlated subqueries             |
 | [Unions and CTEs](./docs/unions-and-ctes.md)               | `unionAll`, `fromUnion`, `withCTE`                              |

docs/README.md

@@ -49,7 +49,7 @@ Roughly in reading order.
 | ----------------------------------------------------- | ----------------------------------------------------------------------------------- |
 | [Getting started](./getting-started.md)               | Install, define a table, build → compile → decode                                   |
 | [Tables and column types](./tables-and-types.md)      | `table()`, column options, external tables, the column-type constructors            |
-| [Building queries](./queries.md)                      | `select`, `where`, `groupBy`, `orderBy`, `limit`, `format`, immutability            |
+| [Building queries](./queries.md)                      | `select`, `where`, `groupBy`, `orderBy`, `limit`, `format`, immutability, `pipe`    |
 | [Expressions and conditions](./expressions.md)        | Comparisons, arithmetic, optional predicates, aggregates                            |
 | [Joins and subqueries](./joins-and-subqueries.md)     | The join family, `fromQuery`, correlated subqueries                                 |
 | [Unions and CTEs](./unions-and-ctes.md)               | `unionAll`, `fromUnion`, `withCTE`                                                  |

docs/queries.md

@@ -202,6 +202,45 @@ See [Unions and CTEs](./unions-and-ctes.md#ctes).
 `.route("ingest")` and `.crossTenant()` attach metadata to the compiled result rather than
 changing the SQL. Both are covered in [Tenant scoping](./tenant-scoping.md).
 
+## Pipeable, data-last queries
+
+A query is `Pipeable`, and `Query` has a data-last form of each builder method, as `Effect`,
+`Stream` and `Schema` do. Each `Query` function is a `dual` that calls the fluent method, so
+both styles write the same SQL:
+
+```ts
+CH.Query.from(Events).pipe(
+	CH.Query.select(($) => ({ name: $.Name, ms: $.DurationMs })),
+	CH.Query.where(($) => [$.OrgId.eq("org_123")]),
+	CH.Query.orderBy(["ms", "desc"]),
+	CH.Query.limit(10),
+)
+// the same SQL as CH.from(Events).select(…).where(…).orderBy(…).limit(10)
+```
+
+`$` in a data-last callback is typed from the query that `pipe` carries, and a wrong column or
+an unselected `orderBy` key is a type error at that step. Each function also has a data-first
+form: `CH.Query.where(query, ($) => […])`.
+
+`Query` has `from`, `fromQuery` and `fromUnion` to start, and a function for every `CHQuery`
+method: `select`, `where`, `groupBy`, `having`, `orderBy`, `limit`, `offset`, `format`,
+`distinct`, `distinctOn`, the row locks (`forUpdate`, `forNoKeyUpdate`, `forShare`,
+`forKeyShare`), `route`, `crossTenant`, the joins (`innerJoin`, `leftJoin`, `crossJoin` and
+their `…Query` forms) and `withCTE`. A form a dialect lacks (a row lock on ClickHouse or SQLite,
+`format` outside ClickHouse, `distinctOn` on SQLite) is refused at compile, as the fluent method
+is. Unions (`CHUnionQuery`) and writes (`insertInto`, `update`, `deleteFrom`) are not pipeable
+yet; use their fluent methods.
+
+A reusable part is a plain function. Type its `$` with the table's columns:
+
+```ts
+const byOrg = (orgId: string) => CH.Query.where(($: CH.ColumnAccessor<typeof Events.columns>) => [$.OrgId.eq(orgId)])
+
+CH.Query.from(Events).pipe(byOrg("org_123"), CH.Query.select("Name"))
+```
+
+_(Backed by `docs/queries.md > Pipeable, data-last queries`.)_
+
 ## Compiling
 
 ```ts

docs/reference.md

@@ -73,6 +73,32 @@ Tables come from the dialect's `table`: [`CH.table`](#tables-and-ddl) or
 
 `CHUnionQuery` offers only `orderBy`, `limit`, `offset`, `format`.
 
+A `CHQuery` is `Pipeable`: `query.pipe(f, g)` is `g(f(query))`.
+
+### `Query` (data-last)
+
+`Query` is a namespace of `dual` functions over the `CHQuery` methods above. Each one is
+data-last (`query.pipe(Query.where(fn))`) and data-first (`Query.where(query, fn)`), calls the
+fluent method, and writes the same SQL. See [Pipeable, data-last queries](./queries.md#pipeable-data-last-queries).
+
+| Export                                                    | Same as                         |
+| --------------------------------------------------------- | ------------------------------- |
+| `Query.from` / `Query.fromQuery` / `Query.fromUnion`      | `from` / `fromQuery` / `fromUnion` |
+| `Query.select()` / `Query.select(...names)` / `Query.select(fn)` | `select`                 |
+| `Query.where(fn)`                                         | `where`                         |
+| `Query.orderBy(...[key, dir])` / `Query.orderBy(fn)`      | `orderBy`                       |
+| `Query.limit(n)` / `Query.offset(n)`                      | `limit` / `offset`              |
+| `Query.groupBy(...keys)` / `Query.having(fn)`             | `groupBy` / `having`            |
+| `Query.distinct()` / `Query.distinctOn(...keys)`          | `distinct` / `distinctOn`       |
+| `Query.format(fmt)`                                       | `format`                        |
+| `Query.forUpdate` / `Query.forNoKeyUpdate` / `Query.forShare` / `Query.forKeyShare` | `(options?)`, the row locks |
+| `Query.route(route)` / `Query.crossTenant()`              | `route` / `crossTenant`         |
+| `Query.innerJoin` / `Query.leftJoin` / `Query.crossJoin`  | `(table, alias, on?)`           |
+| `Query.innerJoinQuery` / `Query.leftJoinQuery` / `Query.crossJoinQuery` | `(query, alias, on?)` |
+| `Query.withCTE(name, query)` / `Query.withCTE(name, sql, options?)` | `withCTE`             |
+
+`CHUnionQuery` and the writes are not `Pipeable`; use their methods.
+
 ### Compilation
 
 Each entry exports `compile`, `compileUnsafe`, `compileUnion` and `compileUnionUnsafe`, defaulting

src/ch/query-pipe.test-d.ts

@@ -0,0 +1,135 @@
+// Type-level tests: the pipeable, data-last query API types its callbacks and
+// its result as the fluent builder does.
+
+import { expectTypeOf } from "expect-type"
+import { pipe } from "effect"
+import * as CH from "./index"
+import type { InferQueryOutput } from "./query"
+
+const Tours = CH.table("tours", { OrgId: CH.string, Id: CH.string, StartsAt: CH.dateTime64, Notes: CH.nullable(CH.string) })
+const Units = CH.table("units", { Id: CH.string, TourId: CH.string, Name: CH.string })
+
+const { Query } = CH
+
+// The issue's example: `$` is typed from the query `pipe` carries.
+const piped = Query.from(Tours).pipe(
+	Query.select(($) => ({ id: $.Id, at: $.StartsAt })),
+	Query.where(($) => [$.OrgId.eq(CH.param.string("orgId"))]),
+	Query.orderBy(["at", "asc"]),
+	Query.limit(10),
+)
+const fluent = CH.from(Tours)
+	.select(($) => ({ id: $.Id, at: $.StartsAt }))
+	.where(($) => [$.OrgId.eq(CH.param.string("orgId"))])
+	.orderBy(["at", "asc"])
+	.limit(10)
+expectTypeOf(piped).toEqualTypeOf(fluent)
+expectTypeOf<InferQueryOutput<typeof piped>>().toEqualTypeOf<{ readonly id: string; readonly at: InferQueryOutput<typeof fluent>["at"] }>()
+
+// The param a data-last `where` adds is a param of the query.
+// @ts-expect-error -- `orgId` is missing
+CH.compileUnsafe(piped, {})
+CH.compileUnsafe(piped, { orgId: "org_1" })
+
+// Data-first gives the same type.
+expectTypeOf(Query.where(CH.from(Tours).select("Id"), ($) => [$.OrgId.eq("o")])).toEqualTypeOf(
+	CH.from(Tours)
+		.select("Id")
+		.where(($) => [$.OrgId.eq("o")]),
+)
+
+// A reusable part is a plain function. Its `$` is typed from the table it names,
+// and the query keeps every column, so a later step can read any of them.
+const byOrg = (orgId: string) => Query.where(($: CH.ColumnAccessor<typeof Tours.columns>) => [$.OrgId.eq(orgId)])
+const reused = Query.from(Tours).pipe(byOrg("org_1"), Query.select("Id", "StartsAt"))
+const reusedFluent = CH.from(Tours)
+	.where(($) => [$.OrgId.eq("org_1")])
+	.select("Id", "StartsAt")
+export const reusedIsFluent: typeof reusedFluent = reused
+export const fluentIsReused: typeof reused = reusedFluent
+
+// `pipe` from effect works too.
+expectTypeOf(pipe(CH.from(Tours), Query.select("Id", "Notes"))).toEqualTypeOf(CH.from(Tours).select("Id", "Notes"))
+expectTypeOf(pipe(CH.from(Tours), Query.select())).toEqualTypeOf(CH.from(Tours).select())
+
+// An unknown column in a data-last callback is a type error.
+CH.from(Tours).pipe(
+	// @ts-expect-error -- `Nope` is not a column of `tours`
+	Query.where(($) => [$.Nope.eq("x")]),
+)
+// @ts-expect-error -- `Nope` is not a column of `tours`
+CH.from(Tours).pipe(Query.select("Nope"))
+
+// orderBy by a key the query does not select is a type error.
+CH.from(Tours).pipe(
+	Query.select("Id"),
+	// @ts-expect-error -- `at` is not selected
+	Query.orderBy(["at", "asc"]),
+)
+// @ts-expect-error -- `asc` or `desc` only
+CH.from(Tours).select("Id").pipe(Query.orderBy(["Id", "up"]))
+// @ts-expect-error -- a negative limit
+CH.from(Tours).select("Id").pipe(Query.limit(-1))
+
+// Joins: `$.<alias>` reads the joined table; leftJoin makes it nullable.
+const joined = CH.from(Tours).pipe(
+	Query.innerJoin(Units, "u", (t, u) => t.Id.eq(u.TourId)),
+	Query.select(($) => ({ id: $.Id, unit: $.u.Name })),
+)
+expectTypeOf(joined).toEqualTypeOf(
+	CH.from(Tours)
+		.innerJoin(Units, "u", (t, u) => t.Id.eq(u.TourId))
+		.select(($) => ({ id: $.Id, unit: $.u.Name })),
+)
+const left = CH.from(Tours).pipe(
+	Query.leftJoin(Units, "u", (t, u) => t.Id.eq(u.TourId)),
+	Query.select(($) => ({ unit: $.u.Name })),
+)
+expectTypeOf<InferQueryOutput<typeof left>>().toEqualTypeOf<{ readonly unit: string | null }>()
+const crossed = CH.from(Tours).pipe(Query.crossJoin(Units, "u"), Query.select(($) => ({ unit: $.u.Name })))
+expectTypeOf<InferQueryOutput<typeof crossed>>().toEqualTypeOf<{ readonly unit: string }>()
+
+CH.from(Tours).pipe(
+	Query.innerJoin(Units, "u", (t, u) => t.Id.eq(u.TourId)),
+	// @ts-expect-error -- the alias `u` is in use
+	Query.innerJoin(Units, "u", (t, u) => t.Id.eq(u.TourId)),
+)
+
+// The rest of the builder: each data-last form gives the fluent type.
+const ids = CH.from(Tours).select(($) => ({ id: $.Id, at: $.StartsAt }))
+expectTypeOf(ids.pipe(Query.groupBy("id"), Query.having(($) => [$.OrgId.eq(CH.param.string("orgId"))]))).toEqualTypeOf(
+	ids.groupBy("id").having(($) => [$.OrgId.eq(CH.param.string("orgId"))]),
+)
+expectTypeOf(ids.pipe(Query.offset(5), Query.distinct(), Query.format("JSON"))).toEqualTypeOf(ids.offset(5).distinct().format("JSON"))
+expectTypeOf(ids.pipe(Query.distinctOn("id"), Query.crossTenant())).toEqualTypeOf(ids.distinctOn("id").crossTenant())
+expectTypeOf(ids.pipe(Query.forUpdate(), Query.forShare({ skipLocked: true }))).toEqualTypeOf(ids.forUpdate().forShare({ skipLocked: true }))
+expectTypeOf(Query.forKeyShare(ids)).toEqualTypeOf(ids.forKeyShare())
+expectTypeOf(ids.pipe(Query.route("replica"))).toEqualTypeOf(ids.route("replica"))
+// @ts-expect-error -- `nope` is not selected
+ids.pipe(Query.groupBy("nope"))
+// @ts-expect-error -- `nope` is not selected
+ids.pipe(Query.distinctOn("nope"))
+// @ts-expect-error -- a negative offset
+ids.pipe(Query.offset(-1))
+
+// Subquery joins and CTEs carry the inner query's params.
+const units = CH.from(Units)
+	.select(($) => ({ tour: $.TourId, name: $.Name }))
+	.where(($) => [$.Name.eq(CH.param.string("unit"))])
+const viaUnits = CH.from(Tours).pipe(
+	Query.leftJoinQuery(units, "u", (t, u) => t.Id.eq(u.tour)),
+	Query.select(($) => ({ id: $.Id, unit: $.u.name })),
+)
+expectTypeOf<InferQueryOutput<typeof viaUnits>>().toEqualTypeOf<{ readonly id: string; readonly unit: string | null }>()
+// @ts-expect-error -- `unit` is missing
+CH.compileUnsafe(viaUnits, {})
+expectTypeOf(CH.from(Tours).pipe(Query.innerJoinQuery(units, "u", (t, u) => t.Id.eq(u.tour)))).toEqualTypeOf(
+	CH.from(Tours).innerJoinQuery(units, "u", (t, u) => t.Id.eq(u.tour)),
+)
+expectTypeOf(CH.from(Tours).pipe(Query.crossJoinQuery(units, "u"))).toEqualTypeOf(CH.from(Tours).crossJoinQuery(units, "u"))
+expectTypeOf(ids.pipe(Query.withCTE("u", units))).toEqualTypeOf(ids.withCTE("u", units))
+expectTypeOf(ids.pipe(Query.withCTE("one", "SELECT 1"))).toEqualTypeOf(ids.withCTE("one", "SELECT 1"))
+CH.from(Tours).pipe(
+	// @ts-expect-error -- the subquery selects no `nope`
+	Query.innerJoinQuery(units, "u", (t, u) => t.Id.eq(u.nope)),
+)