Require the company renormalize pass in the re-key ceremony - #290
Open
sroussey wants to merge 1 commit into
Open
Require the company renormalize pass in the re-key ceremony#290sroussey wants to merge 1 commit into
sroussey wants to merge 1 commit into
Conversation
The two `truncate-identity-tier` scripts justified sparing the company canonical tier by asserting `normalizeCompanyName` is unchanged. That claim was false as of the very merge that shipped them: the same release fixed the placeholder-suffix regex bug, so `Churchill Capital Corp I` no longer normalizes onto `Churchill Capital`, `Reinvent Technology Partners Y` no longer collides with `Reinvent Technology Partners`, and `Blue Acquisition Corp/Cayman` now reaches the legal form behind EDGAR's jurisdiction suffix. The tier should still be spared — the rows are rebuildable, not disposable — but the ceremony has to prescribe the rebuild. It does not happen on its own and nothing errors when it is skipped. - both SQL scripts: rewrite the COMPANY paragraph with the concrete before/after pairs, require `sec resolve --kind company --all --renormalize` as step 3b (after the backfills, before the alias imports, which match on canonical display names), and state the zero-link canonical residue an operator will see afterwards. - postgres variant: `quote_ident(current_schema())` in the `search_path` pin. `search_path` parses its elements as identifiers, so an unquoted `Staging` is written verbatim and resolved as `staging` — the pin silently fails open on exactly the multi-schema deployment it exists to protect. - truncateIdentityTier.test.ts: widen the `set_config` regex to admit the wrapper (keeping the negative assertion on the `SET ... TO` spelling Postgres rejects), pin `quote_ident` directly, and add two tests requiring the renormalize command and forbidding the stale "unchanged" claim in both variants. - CompanyNormalization.ts: the `generateCompanyHash` JSDoc said the prerequisite was teaching `ResolveObservationsTask` to re-normalize. That shipped; say so, and say why the fold is still not applied. - canonical.ts: `alias-list` and `suggest-aliases` fell through to text for any unrecognized `--format`, so `--format json` produced a dump `alias-import` cannot read — discovered only after the wipe the export was taken for. Validate against text|tsv before querying. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Both
truncate-identity-tierscripts justify sparing the company canonical tier by asserting thatnormalizeCompanyNameis unchanged. That claim was false as of the very merge that shipped the scripts. The same release fixed the placeholder-suffixRegExpbug (a[related person is an entity]literal interpolated into a pattern, where its brackets are a character class), which moved the company key for real names:Churchill Capital Corp IChurchill CapitalChurchill Capital Corp IReinvent Technology Partners YReinvent Technology PartnersBlue Acquisition Corp/Cayman/Cayman)Blue AcquisitionGores Holdings X, Inc. / CIGores Holdings X Inc / CIGores Holdings XThe first two are the point of that release:
Reinvent Technology Partners(CIK 1819848, now Joby) andReinvent Technology Partners Y(CIK 1828108, now Hippo) are two different companies that shared one canonical identity.An operator following the ceremony today reads "the company tier is untouched, nothing to do here", runs the wipe and the backfills, and keeps every merged canonical identity the release exists to split — silently. Nothing errors: the stale
normalized_namevalues keep resolving, andversion coveragekeeps reporting full coverage.The fix
The tier should still be spared from the wipe — those rows are rebuildable, not disposable.
normalized_namederives from thenameevery company observation already carries, so one command recomputes it in place. What was missing is that the ceremony has to require that command.scripts/sql/truncate-identity-tier.sql— rewrite the COMPANY paragraph: the tier is not wiped, but it is not untouched either. Concrete before/after pairs, the reasoning (rebuildable, so wiping would cost a re-extraction and its AI bill for a value one command recomputes), and thesec resolve --kind company --all --renormalize→ 3c curated imports. The renormalize sits before the alias imports on purpose: aliases match on canonical display names, so they must land against the re-resolved tier.canonical_companyrows minted under the previous normalized names survive the re-partition with zero identity links. They are inert, not corruption — nothing reads a canonical row no link cites. The visible fallout is aliases whose target became one of them, whichsec canonical company alias-list --orphanslists.scripts/sql/truncate-identity-tier.postgres.sql— same correction inside theTRUNCATEblock, pointing at the portable file for the ordered command list. Plus a separate low-severity fix at thesearch_pathpin:quote_ident(current_schema()).search_pathparses its elements as identifiers, so an unquoted element is case-folded — a schema namedStagingis written verbatim, resolved asstaging, and the pin fails open on exactly the multi-schema deployment it exists to protect.quote_identonly wraps when wrapping is needed, so ordinary lower-case schemas are unaffected.src/storage/versioning/truncateIdentityTier.test.ts— thequote_identchange breaks the pinnedset_configregex (it requiredcurrent_schema()immediately after the comma), so that is widened to admit a wrapper while keeping the negative assertion on theSET … TO current_schema(spelling Postgres rejects. A new test pinsquote_identdirectly. Two more requiresec resolve --kind company --all --renormalizein both variants and forbid the stale "is unchanged" claim. (These read the raw file, sincestatements()strips--lines by design and the instruction is prose.)COMPANY_TIER_TABLESand the "leaves the company canonical tier alone" test stay — the tier is still spared; only the doc comment explaining why was wrong.CLAUDE.md— the "Re-keying without a version bump" section rewritten to match, including the stalesec resolve --kind company --all(missing--renormalize).Included low-severity fixes
CompanyNormalization.ts— thegenerateCompanyHashJSDoc said the prerequisite for folding diacritics innormalizeCompanyNamewas "teachingResolveObservationsTaskto re-normalize as it re-partitions". That shipped. Now says the prerequisite exists, and that the fold is still deliberately unapplied because a command existing is not the same as an operator having run it — turning it on re-keys on the next extraction whether or not stored rows were re-partitioned.CompanyNormalization.test.tsstill pins the gap. (CLAUDE.md was already correct on this point and is left alone.) No copyright-year bump.src/cli/groups/canonical.ts—alias-listandsuggest-aliasesboth didif (format === "tsv") … else text, so--format jsonsilently produced a human-readable dump. Both now validate againsttext|tsvwithprintError+return, matchingsuggest-aliases's existing--kindvalidation. Silent fall-through is worst here specifically: these exports are taken immediately before a destructive re-key, and a text dump is not somethingalias-importcan parse — the operator finds out after the wipe, with the hand-curated alias rows already gone. Validation runs before the query so no partial listing reaches stdout.A note on cost
This fix makes an operator do more work, not less. The renormalize pass is cheap — no fetches, no model calls; it recomputes
normalized_namefrom names already stored — but it does re-partition the company canonical tier, and the zero-link canonical rows left behind should not be read as corruption.Verification
The before/after normalization values above were re-verified by running
normalizeCompanyNameatorigin/maindirectly, not read from the review.Generated by Claude Code