[artile]

Prisma-style cursor/take pages and OFFSET pages have no typed helper in effect-orm. OFFSET reads and drops every earlier row, and on D1 that costs billed rows read.

Closes #86.
Part of #52.

Changes

• /database adds paginate(query, { orderBy, after, limit }, params?). It returns { rows, nextCursor: Option<string> }.
• It adds a seek on the sort key, the ORDER BY and LIMIT limit + 1. The extra row only tells if there is a next page. The query keeps its own where, so the tenant filter and params still apply.
• One direction: a row comparison, ("at", "id") < ($2, $3). Postgres serves it from an index on (at, id); ClickHouse compares it as a tuple.
• Mixed directions: the expanded form, a >= x AND (a > x OR (a = x AND b < y)). The leading bound keeps the seek on the first index column.
• The key is one or more selected aliases. A nullable alias, an alias that is not selected, or an empty key is a type error.
• The cursor is base64url of { k, v }: k names the sort key, v holds the key values encoded through Schema.toCodecJson of the key columns' types. A DateTime.Utc key comes back as the same instant.
• New error InvalidCursor (in the error channel): the cursor does not decode, or it was written for another orderBy.
• limit that is not a positive integer is a QueryBuilderError. A query with offset, groupBy, having or distinctOn, or a key with no type, is a QueryBuilderDefect.
• Internal: operandOf(expr, value) in ch/expr.ts writes a value as the column's comparisons do, for the row comparison. selectExprsOf and selectRecordOf in ch/compile.ts are now exported (internal, not on an entry).
• Docs: a Pagination section in docs/database.md, the reference rows, and the cursor note in docs/recipes.md. One minor changeset.

Left out

• SQLite, MySQL and D1: there is no dialect for them on main. The row comparison is valid SQLite and MySQL; a dialect PR can add tests.
• A check that the key is unique: a table does not say which aliases of a query are unique. The docs and the option doc say to end the key with the primary key.
• Grouped queries (groupBy / having): the seek would have to go in HAVING. Refused for now.
• Backward pages (before) and a total count: not needed for a first version.
• A signed or encrypted cursor: it needs a key the library does not have. The cursor only moves a reader inside its own where, so a changed cursor cannot cross the tenant filter. The docs say not to put secret values in the key.
• A pipeable Query.paginate: the pipe API is in feat(query): a pipeable, data-last Query API next to the fluent builder #102. paginate is a function over the Database in context, not a DatabaseApi method, to keep the change small.

Validation

• bun run build: ok.
• bun run typecheck: ok.
• bunx vitest run --testTimeout=60000 --hookTimeout=60000 with EFFECT_ORM_CLICKHOUSE_URL set (ClickHouse 26.8.2.7): 62 files, 1046 tests passed, 0 skipped. New: src/database/paginate.test.ts (10, PGlite), src/database/paginate.test-d.ts, tests/paginate.clickhouse.test.ts (2, live ClickHouse).
• node scripts/check-doc-citations.mjs, node scripts/check-doc-examples.mjs, node scripts/check-exports-documented.mjs: ok.
• bun scripts/check-package.ts: ok.
• Upstream CI for a fork PR waits for a maintainer to approve the run.

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 47 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 0fb8adb9-d772-4dce-a11f-f5d627a7acdf

Commits

Reviewing files that changed from the base of the PR and between b57faf1 and 46e517f.

Files selected for processing: 13

• .changeset/keyset-pagination.md
• docs/database.md
• docs/recipes.md
• docs/reference.md
• src/ch/compile.ts
• src/ch/expr.ts
• src/database.ts
• src/database/errors.ts
• src/database/paginate.sqlite.test.ts
• src/database/paginate.test-d.ts
• src/database/paginate.test.ts
• src/database/paginate.ts
• tests/paginate.clickhouse.test.ts

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[Makisuo]

Merge-readiness pass (46e517f):

• Merged current main. The only conflict was docs/reference.md; both sides are kept (paginate/InvalidCursor and afterCommit).
• NULL ordering: dialects disagree here. Ascending, SQLite and D1 put NULLs first; Postgres and ClickHouse put them last. A NULL key also drops out of a row comparison. The PR already refuses nullable keys at the type level. paginate now refuses them at run time too, with a QueryBuilderDefect, when the key's schema accepts null (a cast or untyped caller). Docs and changeset updated.
• SQLite/D1: the new src/database/paginate.sqlite.test.ts walks every page on node:sqlite and on a Miniflare D1 with requireTenantScope. It covers the row-comparison seek (desc/desc), the mixed-direction seek (asc/desc) and SQLite.timestamp keys stored as text, which round-trip through the cursor. Ties are broken by id, and every row comes back exactly once. No library change was needed for these dialects.

Verified locally: bun run typecheck passes. bun run test against live ClickHouse 26.8 passes (76 files, 1444 tests), including tests/paginate.clickhouse.test.ts. FUZZ_SCALE=0.3 vitest run tests/fuzz passes (9 files, 269 tests). CI run 41473163801 started without needing approval.