fix(rekey): scope the truncate scripts, and make curated aliases restorable

[sroussey]

Stacked on #280 — its decisions are what this PR's script headers assert (the company tier is not stale; the family key is final). Review/merge that one first.

Four corrections to the documented re-key ceremony. Each one costs an operator data or money today.

1. The scripts wiped the company canonical tier on a wrong rationale

The header justified it with "generateCompanyHash folds … so company_hash_id changes for those names". No table stores company_hash_id. canonical_company is keyed by a UUID plus normalized_name, produced by normalizeCompanyName — unchanged.

So an operator following the documented ceremony destroyed canonical_company, canonical_company_alias, canonical_company_address, canonical_company_phone and company_identity_link, none of which the normalizer change made stale — and with no rebuild path. sec resolve --kind company --all re-resolves from the company observations these scripts keep, so it can only rebuild links that were never stale.

That whole group is gone from both files, and the header now names the real stale set: the person normalized_* / person_hash_id generation, and the family normalized_name keys.

observation_provenance is scoped WHERE kind = 'person' for the same reason — its kind: "company" rows (underwriter and issuer observations) cite observations that survive. On Postgres that means moving it out of the TRUNCATE list into a scoped DELETE inside the same transaction.

2. The re-extraction gates re-paid AI cost for filings nothing touched

extractor_runs and extraction_dead_letter were cleared wholesale, so the forms sweep re-selected every filing — re-running 8-K redemption/LOI detection and merger-proxy extraction, whose output the script never deleted. Both deletes are now scoped to a new PERSON_OBSERVING_EXTRACTOR_IDS (src/storage/versioning/extractorIds.ts), derived from the observePerson call sites: S-1 D C CFPORTAL 1-A 1-Z 3 4 5 144. A test asserts the SQL and the constant cannot drift.

3. The portable script advertised psql against unqualified names

Its usage block listed psql "$SEC_PG_URL" -f …truncate-identity-tier.sql next to the sqlite3 line while every DELETE FROM is unqualified — so on a deployment whose search_path lists a staging schema first, following the file's own instructions deletes the other schema's identity tier, irreversibly.

The psql line is removed rather than SET LOCAL search_path added: sqlite3 rejects that statement, and portability is this file's only reason to exist. Postgres users are pointed at the .postgres.sql variant (which already pins the schema), with the reason stated in the comment. Enforced by test.

4. Aliases were destroyed with no way to restate them

Alias rows are hand-curated claims keyed by canonical UUIDs the wipe itself destroys — so unlike family_description they cannot be spared. But the only dump surface printed alias_canonical_id -> target_canonical_id and never the names, so after the wipe an operator holding that output could reconstruct nothing: the UUIDs no longer resolve.

All four alias-list commands (person, company, sponsor-family, underwriter-family) now:

• print resolved display names alongside the ids;
• accept --format tsv, the machine-readable export;
• gain a matching sec canonical <kind> alias-import <file.tsv>.

TSV, not CSV, deliberately. These files hold names that routinely contain commas — Keefe, Bruyette & Woods, Inc., Frank R. Martire, Jr. — and the repo's only CSV reader (editorialImport.ts) splits on commas with no quoting, so a CSV export would be silently unreadable for exactly the names most worth restoring.

Import locates columns by header name (a reordered or trimmed file still works), resolves each pair by name, and reports a pair it cannot place without abandoning the rest — after a wipe, a name whose canonical row has not been re-extracted yet is an expected partial failure, not a reason to lose the other forty.

The two family groups now share one registration helper (familyAliasExchange.ts), so the export an operator takes before a wipe has the same shape on both — a wipe hits both at once.

Tests

src/storage/versioning/truncateIdentityTier.test.ts reads both .sql files as text and pins: the extractor-id list matches the constant; neither file names a company-tier table; the two variants name the same table set; every table named exists in SEC_STORAGE_REGISTRY; provenance is person-scoped; the portable file carries no SET LOCAL search_path and no psql usage line while the Postgres one does pin the schema.

Verified red against the pre-fix scripts: 7 of 10 failed, 10/10 pass after.

Note: it lives under src/ rather than beside the scripts because vitest's include is src/**/*.test.ts — a test in scripts/ would silently never run. Flagging in case you'd rather widen the include.

src/task/canonical/aliasExchange.test.ts runs the round trip end to end: curate an alias, export, wipe (alias rows and the canonical rows they cite), re-mint the canonicals under new ids, re-import from the export, assert the set is restored against the new ids. Plus the family-tier equivalent, orphan rendering, and the parser's header/missing-name behavior. Verified red by neutering displayNames to an empty map (3 of 6 fail).

src/cli/groups/canonical.test.ts gains an end-to-end check that alias-list --format tsv emits the header alias-import reads.

Verified

npx tsc --noEmit clean. Green: truncateIdentityTier.test.ts 10/10, aliasExchange.test.ts 6/6, canonical.test.ts 18/18, commandsBoot.test.ts 19/19, version.test.ts 12/12, src/storage/company + src/resolver 64/64.

Note on version.test.ts flakiness under load

While validating, runs of src/cli/groups produced failures in version.test.ts — a file this PR does not touch. Every failure across every run was Error: Test timed out in 15000ms, never an assertion failure, and the count tracked host load, which ranged 7–53 from an unrelated concurrent job:

host load	failures
53	19 (across src/config, src/commands, src/cli/groups)
26	8 (7 version, 1 resolve)
11–16	3 (all version)
10–20	2 (all version)

Resolved by a like-for-like A/B: version.test.ts run alone against an unmodified origin/main worktree and against this branch, back to back at low load —

main    Test Files 1 passed (1)   Tests 12 passed (12)   0 timeouts   67.10s
branch  Test Files 1 passed (1)   Tests 12 passed (12)   0 timeouts   65.94s

Identical, including runtime. Consistent with the startup benchmark, taken interleaved to cancel load drift over the operation each failing test runs first:

db setup   main   1730 1893 1774 1981 1919 1967   mean 1877 ms
           branch 1738 1836 1831 1796 1572 1686   mean 1743 ms

No regression (the branch is marginally faster). An earlier sequential measurement of mine appeared to show a ~200 ms branch slowdown; that was an artifact of running all main samples before all branch samples, and the interleaved run refutes it.

So the failures were contention against a 15 s budget that is tight for tests spending ~3.6–3.9 s per assertion on sec subprocess boots — not a regression from this diff.

---

🤖 Generated with Claude Code