fix(spac): let a completion outrank a deregistration, and order the forms sweep

[sroussey]

Stacked on #281 — base is claude/keen-knuth-hxoj5s-8k-replay, not main. Merge #281 first; GitHub will retarget this to main automatically. Both PRs touch spacDealGrouping.ts, in different functions, so stacking keeps the conflict at zero.

Two Form 25 / deregistration defects.

---

Defect 1 (HIGH) — a deregistration dated at or before the completion turns a completed de-SPAC into "liquidated"

In deriveDeals, the liquidation / deregistration branch set walkTerminal and broke the event walk unconditionally. Events are ordered by (event_date, accession_number), and the two dates come from different clocks: the completed event is dated by the 8-K's report date, while the Form 25 / 25-NSE event is dated by its filing date. So the routine post-close delisting of a de-SPAC'd shell's units routinely collides with — or sorts ahead of — the closing it follows. processDeregistration writes the event for any known SPAC with no check that a completion already exists.

Concrete failure (verified by executing deriveDeals)

Events [definitive_agreement 2022-01-10, deregistration 2022-06-16, completed 2022-06-21] — a 2.01 8-K whose report_date is absent so event_date falls back to filing_date, plus the routine post-close Form 25 delisting the SPAC's units on the closing day. The walk ends at the deregistration and never reads the completion:

• deriveDeals → outcome: "terminated", outcome_date: "2022-06-16", source_accession: "0001-form25"
• buildSpacRow → status: "liquidated", failed_date: 2022-06-16, completed_date: null, surviving_name: null

A successfully de-SPAC'd company is reported as a liquidated shell. The same happens on an exact date tie whenever the Form 25's accession string sorts below the 8-K's.

Approach

Pre-scan the relevant events for a completion, and skip the failure branch entirely when one exists anywhere in the stream:

const hasCompleted = relevant.some((e) => e.event_type === "completed");
// ...
case "liquidation":
case "deregistration": {
  if (!hasCompleted) { /* close the open deal, walkTerminal = true */ }
  break;
}

Deliberately not break-ing: the walk continues and reaches the completed event, which sets walkTerminal itself. A liquidation genuinely after a completion was already unreachable (the completion breaks the walk first), so the guard cannot mask a real post-completion failure.

spacRollup.ts needs no edit — its hasFailed already keys on deals.some(d => d.outcome === "completed") and follows automatically.

Rejected alternatives

• Rank completed ahead of the failure types in the same-date sort. Perturbs deal_index assignment, which is a stability contract the existing test assigns the same deal_index regardless of event insertion order guards. It also only fixes the exact-tie case, not the inverted-date one.
• Change processDeregistration to skip writing the event when a completion exists. The Form 25 is a true fact and belongs in the append-only log. Only its interpretation is wrong, so only the interpretation moves.

---

Defect 2 (MEDIUM) — the forms sweep processes Form 25 before the S-1, so a first-pass sweep drops every deregistration as a successful no-op

ComputeFormsWorklistTask defaults its form list to Object.keys(FORM_TO_EXTRACTOR_ID), and JS enumerates integer-like keys first in ascending numeric order. Executed, that yields ["3","4","5","25","144","D",...] — the bare "25" runs fourth, well ahead of "S-1" (index 33), "424B4" (46) and "8-K" (48). ("25/A", "25-NSE" and the 15-family are non-integer keys and already fall at the end, so only the bare 25 — the common form — is misordered.)

processDeregistration is known-SPAC gated: it returns silently and the run is recorded successful when no spac row exists yet, so the default extractor_runs anti-join never revisits it.

Concrete failure

sec update forms on a freshly bootstrapped database: every issuer-filed Form 25 is processed before its S-1 has minted the spac row, no deregistration event is written, and each filing is recorded successful. Liquidated SPACs keep a stale searching/deal_announced status and a null failed_date indefinitely.

Approach

Make the order explicit rather than incidental. sortFormsForSweep in storage/versioning/extractorIds.ts ranks by a declared SWEEP_PRIORITY of ["S-1", "424", "8-K", "merger-proxy", "25-15"] — a comment states the dependency (registration/prospectus mints the spac row; 8-K, proxies and 25/15 are gated on it). Unranked extractors (D, C, 3/4/5, 144, 1-A, …) follow, and the sort is stable within a rank so S-1 precedes S-1/A precedes DRS.

ComputeFormsWorklistTask applies it to the filtered form list — including an explicit --form list, so a multi-form request is ordered correctly without the operator knowing to do it. A form with no registered extractor is still filtered and warned about by the caller; the sort never drops one.

---

Tests

src/storage/spac/spacDealGrouping.test.ts (new describe):

• does not terminate the deal when the deregistration is dated before the completion — the exact executed stream; asserts completed, outcome_date 2022-06-21, source_accession = the 8-K. Failed before the fix with expected 'terminated' to be 'completed'.
• does not terminate the deal on a same-date Form 25 whose accession sorts first — same failure.
• does not let a post-close liquidation event reopen or fail the completed deal — guard, passes before and after; proves the fix is scoped.
• The existing "dereg/liquidation with no completion anywhere" tests stay green — that is the other half of the scoping guard.

src/storage/spac/spacRollup.test.ts:

• a post-close Form 25 leaves status completed with a null failed_date — asserts status: completed, failed_date: null, completed_date: 2022-06-21, surviving_name still derived from the completed deal's target. It derives the deals through deriveDeals rather than hand-writing them, because the rollup only reads deals.some(completed) — a hand-written completed deal would make the test pass against the unfixed code and prove nothing. Failed before the fix with expected 'liquidated' to be 'completed'.

src/storage/versioning/formsSweepOrder.test.ts (new file):

• runs S-1 before 424 before 8-K before proxies before 25/15 over sortFormsForSweep(Object.keys(FORM_TO_EXTRACTOR_ID)).
• keeps every form exactly once, so a newly wired form cannot be dropped — set-equality plus length.
• Plus rank-boundary, stability, explicit-subset and unregistered-form cases.

src/task/forms/formsSweep.test.ts:

• drains forms in sweep order, not object-key order — seeds one Form 25 and one S-1 filing, runs with no --form, asserts the emitted form column reaches S-1 before 25. Failed before the fix with expected 1 to be less than 0.

Every one of the four defect tests was written first and confirmed failing.

Verification

• npx vitest run src/storage/spac → 14 files, 104 tests passing.
• npx vitest run src/task/forms/formsSweep.test.ts src/storage/versioning/formsSweepOrder.test.ts → 16 tests passing.
• npx vitest run src/storage/spac src/storage/versioning src/sec/forms/miscellaneous-filings src/task/forms src/task/spac src/commands → 59 files, 488 tests, one failure: componentRegistry.test.ts > listRegisteredComponents returns one entry per extractor and resolver expects 20 and gets 21. This is pre-existing on origin/main — confirmed by checking out main's file contents and re-running it — and is untouched by this PR (EXTRACTOR_IDS already has 17 entries against the test's stated 16). Not fixed here to keep the diff scoped.
• npx tsc --noEmit clean; bun run build clean.
• The full 338-file suite was not run to completion in this environment (4 cores, ~40 load average); the directories touched were run in full instead.

Risk

• deriveDeals readers: SpacReportWriter.recomputeAndSaveDeals, sec spac report, and three spacDealGrouping.* test files. The change is a pure narrowing of one branch.
• Shard caveat: ComputeFormsWorklistTask resume state is formPos, an index into the form list — a shard resuming across a deploy that changes the order would resume at a different form. Drain or restart in-flight --shard processes when deploying this.

Operator step (no code migration)

On any database bootstrapped before this fix, liquidated SPACs carry a stale status and a null failed_date. Existing recovery works and just needs running:

sec extractor backfill 25-15   # no --force: its filterTodo already selects known-SPAC
                               # Form 25/15 filings with no deregistration event

For SPACs whose deal was wrongly flipped to terminated by defect 1, deriveDeals only re-runs on a write, so the corrected code needs a trigger. Either the #281 repair pass covers it, or use the cheaper targeted form sec extractor backfill 25-15 --force.

---

🤖 Generated with Claude Code

---

Generated by Claude Code