Repository navigation
feat(seed): seed rows from column Schemas through Schema -> Arbitrary - #113
Conversation
A new /seed entry: rows, insert, arbitrary and pick. Deterministic with a seed, one seed stream per column, storage ranges for the built-in column types, per-column overrides, distinct values for unique columns, the tenant column from one tenant, and inserts in batches through insertInto.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 24 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…9, DDL uniques SQLite.integer decodes through a safe-integer check that Arbitrary.schema could only meet by filtering, so every SQLite integer column failed with SeedError; it now samples the safe-integer range, as int8 does (a SQLite.boolean, also an INTEGER column, is left to its Schema). SQLite.timestamp, stored as text, samples the years 0000-9999 that its text covers instead of clustering at 1970. A SQLite table's one-column primary key and unique columns are distinct, as on Postgres. Tested on node:sqlite (and 40 more seeds off-line) and on a local D1 binding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merge-readiness pass (merge ccac6bd, then 20e55bf, 0646d34 and 76e4514, no rebase):
Verification (local, ClickHouse 26.8):
CI run 41476791340 is green on both ClickHouse jobs. The first attempt failed on 26.8 only with the known fuzz flakes (#131) and a 5 s timeout in Interaction note: if #109 (casing) and #111 (client defaults) merge, the seed still maps uniques through |
…nd trip CI's ClickHouse 26.2 parses Float64 literals with the fast parser, so a seeded double could read back one ulp off and the round trip failed there (26.8 defaults the setting on). Document it in docs/seeding.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
precise_float_parsing did not help: on 26.2 the VALUES parser reads some doubles one ulp off even with it on (checked against a 26.2 server; 26.8 reads them exactly). Drop the setting, compare Ratio within a few ulps and every other column exactly, and correct the note in docs/seeding.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te migrate tests 30s ClickHouse 26.2 returns 3 for `-5 % -4` under UNION ALL (26.8: -1). Record it as a known issue scoped to a negative constant divisor. pg-enums-uniques starts a PGlite per test, which timed out at 5s on CI runners busy with the fuzz suite (#110, #113, #114, #131). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Test and demo data is hand-written today, although each column type already has an Effect
Schemathat knows its values. This PR adds a seed entry that makes rows from those Schemas through Effect'sSchema→Arbitrary.Closes #88.
Part of #52.
Changes
@maple-dev/effect-orm/seedwithrows,insert,arbitrary,pickandSeedError. It usesArbitraryfromeffect@4.0.0. There is no new dependency, andfast-checkis not used at runtime.rows(table, { count, seed })returns decoded rows. The same seed gives the same rows. Each column samples from its own seed (the run's seed plus the column key), so a new column does not change the values of the other columns.float4,DateTime/DateTime64/timestamptzyears, lowercaseuuid, no U+0000 in Postgrestextandjsonb. No lone surrogates, no-0. A branded or refined column gets only values that its Schema accepts.columnsoverrides a column with a value, anArbitrary(pick(ids)for a foreign key), or a function of the row index. A wrong override value is aSeedError.uniquecolumns get distinct values. A Postgres table's one-column primary key andunique: truecolumns are added from its DDL.tenant(all rows in one tenant, so the insert compiles as"single-tenant") orcrossTenant: true(the insert declarescrossTenant()).insert(table, options)writes the rows throughinsertInto(table).values(...)on theDatabasein context,batchSizerows per statement (default 100), and returns them.arbitrary(table)gives whole rows toArbitrary.checkEffectfor property tests.chDateTime64Literalinsrc/ch/types.tsis now exported inside the package (not from a public entry).docs/seeding.md, rows indocs/README.md,docs/reference.mdandREADME.md. One changeset (minor).Left out
seed(schema)over many tables). This PR seeds one table at a time; a child table picks its keys from the parent rows withpick. The graph walk can come next if you want it.drizzle-seed. Overrides cover this.effect-orm seedCLI command. The issue names it, but the library API comes first.Validation
All commands ran on this branch, with a live ClickHouse 26.8.2.7 (
EFFECT_ORM_CLICKHOUSE_URL,_USER,_PASSWORDset):bun run build: pass.bun run typecheck: pass.bunx vitest run --testTimeout=60000 --hookTimeout=60000: 62 files passed, 1053 tests passed, 0 skipped.node scripts/check-doc-citations.mjs,node scripts/check-exports-documented.mjs,node scripts/check-doc-examples.mjs: pass.bun scripts/check-package.ts: pass (14 entries load from the tarball under CommonJS).src/seed/seed.test.ts(offline rows, and inserts on PGlite that read back as seeded for every built-in Postgres type),src/seed/seed.test-d.ts(row and override types),tests/seed.clickhouse.test.ts(every built-in ClickHouse type on a live server, rows read back as seeded). I also ran the live insert tests with 30+ other seeds; all passed.Upstream CI for a fork PR waits until a maintainer approves the run.
Need help on this PR? Tag
@codesmith-botwith what you need.