Skip to content

Require the company renormalize pass in the re-key ceremony - #290

Open
sroussey wants to merge 1 commit into
mainfrom
claude/keen-knuth-onotd9-rekey-company-truth
Open

Require the company renormalize pass in the re-key ceremony#290
sroussey wants to merge 1 commit into
mainfrom
claude/keen-knuth-onotd9-rekey-company-truth

Conversation

@sroussey

Copy link
Copy Markdown
Contributor

The problem

Both truncate-identity-tier scripts justify sparing the company canonical tier by asserting that normalizeCompanyName is unchanged. That claim was false as of the very merge that shipped the scripts. The same release fixed the placeholder-suffix RegExp bug (a [related person is an entity] literal interpolated into a pattern, where its brackets are a character class), which moved the company key for real names:

name before after
Churchill Capital Corp I Churchill Capital Churchill Capital Corp I
Reinvent Technology Partners Y Reinvent Technology Partners stays distinct from the plain name
Blue Acquisition Corp/Cayman unchanged (kept /Cayman) Blue Acquisition
Gores Holdings X, Inc. / CI Gores Holdings X Inc / CI Gores Holdings X

The first two are the point of that release: Reinvent Technology Partners (CIK 1819848, now Joby) and Reinvent Technology Partners Y (CIK 1828108, now Hippo) are two different companies that shared one canonical identity.

An operator following the ceremony today reads "the company tier is untouched, nothing to do here", runs the wipe and the backfills, and keeps every merged canonical identity the release exists to split — silently. Nothing errors: the stale normalized_name values keep resolving, and version coverage keeps reporting full coverage.

The fix

The tier should still be spared from the wipe — those rows are rebuildable, not disposable. normalized_name derives from the name every company observation already carries, so one command recomputes it in place. What was missing is that the ceremony has to require that command.

  • scripts/sql/truncate-identity-tier.sql — rewrite the COMPANY paragraph: the tier is not wiped, but it is not untouched either. Concrete before/after pairs, the reasoning (rebuildable, so wiping would cost a re-extraction and its AI bill for a value one command recomputes), and the ⚠️ that skipping the pass fails silently. The re-extraction block is now ordered 3a backfills → 3b sec resolve --kind company --all --renormalize3c curated imports. The renormalize sits before the alias imports on purpose: aliases match on canonical display names, so they must land against the re-resolved tier.
  • Also states the residue: canonical_company rows minted under the previous normalized names survive the re-partition with zero identity links. They are inert, not corruption — nothing reads a canonical row no link cites. The visible fallout is aliases whose target became one of them, which sec canonical company alias-list --orphans lists.
  • scripts/sql/truncate-identity-tier.postgres.sql — same correction inside the TRUNCATE block, pointing at the portable file for the ordered command list. Plus a separate low-severity fix at the search_path pin: quote_ident(current_schema()). search_path parses its elements as identifiers, so an unquoted element is case-folded — a schema named Staging is written verbatim, resolved as staging, and the pin fails open on exactly the multi-schema deployment it exists to protect. quote_ident only wraps when wrapping is needed, so ordinary lower-case schemas are unaffected.
  • src/storage/versioning/truncateIdentityTier.test.ts — the quote_ident change breaks the pinned set_config regex (it required current_schema() immediately after the comma), so that is widened to admit a wrapper while keeping the negative assertion on the SET … TO current_schema( spelling Postgres rejects. A new test pins quote_ident directly. Two more require sec resolve --kind company --all --renormalize in both variants and forbid the stale "is unchanged" claim. (These read the raw file, since statements() strips -- lines by design and the instruction is prose.) COMPANY_TIER_TABLES and the "leaves the company canonical tier alone" test stay — the tier is still spared; only the doc comment explaining why was wrong.
  • CLAUDE.md — the "Re-keying without a version bump" section rewritten to match, including the stale sec resolve --kind company --all (missing --renormalize).

Included low-severity fixes

  • CompanyNormalization.ts — the generateCompanyHash JSDoc said the prerequisite for folding diacritics in normalizeCompanyName was "teaching ResolveObservationsTask to re-normalize as it re-partitions". That shipped. Now says the prerequisite exists, and that the fold is still deliberately unapplied because a command existing is not the same as an operator having run it — turning it on re-keys on the next extraction whether or not stored rows were re-partitioned. CompanyNormalization.test.ts still pins the gap. (CLAUDE.md was already correct on this point and is left alone.) No copyright-year bump.
  • src/cli/groups/canonical.tsalias-list and suggest-aliases both did if (format === "tsv") … else text, so --format json silently produced a human-readable dump. Both now validate against text|tsv with printError + return, matching suggest-aliases's existing --kind validation. Silent fall-through is worst here specifically: these exports are taken immediately before a destructive re-key, and a text dump is not something alias-import can parse — the operator finds out after the wipe, with the hand-curated alias rows already gone. Validation runs before the query so no partial listing reaches stdout.

A note on cost

This fix makes an operator do more work, not less. The renormalize pass is cheap — no fetches, no model calls; it recomputes normalized_name from names already stored — but it does re-partition the company canonical tier, and the zero-link canonical rows left behind should not be read as corruption.

Verification

$ bun run test -- src/storage/versioning/truncateIdentityTier.test.ts \
    src/cli/groups/canonical.test.ts src/storage/company/CompanyNormalization.test.ts

 Test Files  3 passed (3)
      Tests  75 passed (75)
   Duration  116.49s

The before/after normalization values above were re-verified by running normalizeCompanyName at origin/main directly, not read from the review.


Generated by Claude Code

The two `truncate-identity-tier` scripts justified sparing the company
canonical tier by asserting `normalizeCompanyName` is unchanged. That
claim was false as of the very merge that shipped them: the same release
fixed the placeholder-suffix regex bug, so `Churchill Capital Corp I` no
longer normalizes onto `Churchill Capital`, `Reinvent Technology Partners
Y` no longer collides with `Reinvent Technology Partners`, and
`Blue Acquisition Corp/Cayman` now reaches the legal form behind EDGAR's
jurisdiction suffix.

The tier should still be spared — the rows are rebuildable, not
disposable — but the ceremony has to prescribe the rebuild. It does not
happen on its own and nothing errors when it is skipped.

- both SQL scripts: rewrite the COMPANY paragraph with the concrete
  before/after pairs, require `sec resolve --kind company --all
  --renormalize` as step 3b (after the backfills, before the alias
  imports, which match on canonical display names), and state the
  zero-link canonical residue an operator will see afterwards.
- postgres variant: `quote_ident(current_schema())` in the `search_path`
  pin. `search_path` parses its elements as identifiers, so an unquoted
  `Staging` is written verbatim and resolved as `staging` — the pin
  silently fails open on exactly the multi-schema deployment it exists
  to protect.
- truncateIdentityTier.test.ts: widen the `set_config` regex to admit
  the wrapper (keeping the negative assertion on the `SET ... TO`
  spelling Postgres rejects), pin `quote_ident` directly, and add two
  tests requiring the renormalize command and forbidding the stale
  "unchanged" claim in both variants.
- CompanyNormalization.ts: the `generateCompanyHash` JSDoc said the
  prerequisite was teaching `ResolveObservationsTask` to re-normalize.
  That shipped; say so, and say why the fold is still not applied.
- canonical.ts: `alias-list` and `suggest-aliases` fell through to text
  for any unrecognized `--format`, so `--format json` produced a dump
  `alias-import` cannot read — discovered only after the wipe the export
  was taken for. Validate against text|tsv before querying.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants