Skip to content

feat(fast-inbox): detect inbox bucket reorgs by L1 block hash and roll back to the last canonical bucket - #25354

Open
spalladino wants to merge 2 commits into
spl/inbox-descendant-confirmed-eligibilityfrom
spl/inbox-bucket-canonicality-check
Open

feat(fast-inbox): detect inbox bucket reorgs by L1 block hash and roll back to the last canonical bucket#25354
spalladino wants to merge 2 commits into
spl/inbox-descendant-confirmed-eligibilityfrom
spl/inbox-bucket-canonicality-check

Conversation

@spalladino

@spalladino spalladino commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

The archiver now checks that the Inbox buckets it holds sit on L1 blocks that are still canonical, and rolls back to the last canonical bucket when one of them does not.

Context

The archiver decides it is in sync with the Inbox by comparing two numbers against L1: the total message count and the consensus rolling hash (l1_synchronizer.ts, localStateMatches). With the rolling hash covering the messages and nothing else, this reorg is invisible to both:

Fork the node saw                          Canonical fork after the reorg
L1 block 100 (t=1200): A, B → bucket #7    L1 block 100' (t=1200): A, B → bucket #7 (ts 1200)
L1 block 101 (t=1212): C    → bucket #8    L1 block 101' (t=1224): C    → bucket #8 (ts 1224)

Same messages in the same order, so the same count and the same hash — and the existing rollback cannot help either, since it looks for the last message whose rolling hash still matches and they all do. What did change is the metadata the archiver derives from the L1 block: the bucket's timestamp, and (when the reorg merges or splits blocks) its sequence number and boundaries. A bucket's timestamp is what the censorship cutoff is compared against and the recency key the sequencer's bucket selection walks; its sequence number is the bucketHint L1 looks the bucket up with, so a stale one gets the proposal reverted with Rollup__InvalidInboxRollingHash.

#25319 / #25322 / #25327 close this by committing the bucket structure and timestamps into the rolling hash itself, which makes the re-time visible to the check that already exists. This PR is the alternative for keeping the rolling hash over messages only: leave the hash alone and check the bucket metadata directly.

Approach

Each bucket snapshot records the number and hash of the L1 block it was opened in (#25350), and now of the block it was closed in as well. Every sync pass reads those blocks from L1 and compares hashes, before the count-and-hash comparison runs:

  • Buckets are verified upward from a canonicalVerifiedThroughSeq watermark the message store keeps. Buckets at or below it are never looked at again; buckets above it are checked every pass. Rollover siblings share an opening block and so cost a single read.
  • Both ends of a bucket are checked. The Inbox keys buckets by block.timestamp, so where consecutive L1 blocks may share a timestamp (anvil with manual mining) one bucket can absorb messages from several blocks. Reorging only the later one splits the bucket in two while its opening block stays canonical and the message count and rolling hash are untouched — nothing else in the pass would notice. The closing block is store-internal: the check reads bucket spans (InboxBucketL1Span), and the InboxBucket type consumers see is unchanged.
  • On the first mismatch the archiver logs one warn, removes every message absorbed after the last canonical bucket, and rewinds the message syncpoint to just before that bucket's opening L1 block — both in a single store transaction, with the target syncpoint resolved beforehand, so an interruption cannot leave truncated messages behind a syncpoint that would never fetch them again. The normal flow then re-downloads the reorged blocks and rebuilds the snapshots with the timestamps and sequence numbers the canonical chain gives them. Since verification runs upward, the last canonical bucket is simply the previous one — no reverse walk.
  • The check runs before the guard that skips the message sync when L1 has not moved forward, since a reorg can re-mine the head at the same number: by block number nothing advances, so without this the stale syncpoint would be kept and every later pass would short-circuit on it.
  • The watermark advances over a bucket only when both of its stored hashes matched and the block it was closed in was at or below the finalized block reported in that same pass. It is lowered whenever messages are removed, by any path, and whenever a stored bucket is re-delivered from a different L1 block, so a bucket is never trusted on a check made against blocks it no longer sits on.
  • The rolling-hash rollback stays as the fallback for everything this does not resolve (an empty store, a reorg that also changes the messages). Its removal and syncpoint rewind now go through the same atomic path.

"Below finalized" is not a free pass. The existing rollback may stop at the finalized block, because a message at or below it cannot have been reorged while the node was watching. That reasoning does not extend to a node that was not watching: it can hold a block that was orphaned before finalization and has since been buried below it, and no amount of finality will ever make that block canonical. So the watermark records what this node has itself verified, not what the chain has finalized — a bucket sitting below finality that was never checked is still checked once.

RPC cost. One eth_getBlockByNumber per distinct unverified L1 block per sync pass, issued 4 at a time. In the steady state the watermark trails the finalized block, so that is the blocks within the finality window that actually carried messages, deduplicated; on a quiet chain it is one or two, and it stops entirely once the store has nothing above the watermark. If finality stalls the unverified tail grows without bound, so a pass checks at most 32 buckets and the rest wait for the next one — the watermark is a prefix, so progress is never lost. A read that fails ends the pass there and is logged at warn (thinned out when it keeps failing on the same bucket), since a block that cannot be read blocks every bucket above it from ever being verified while the sync itself carries on.

Tests

FakeL1State can now replace a range of L1 blocks with fresh hashes (reorgL1BlocksFrom), re-mine one block's messages into another (retimeMessages, which also covers the merge case), re-mine a block with its messages reordered (reorderMessagesAtL1Block), mine a block carrying an earlier block's timestamp (shareTimestampWithL1Block) and re-mine such a block with a timestamp of its own (splitCoTimestampedL1Block). It also records the L1 blocks read by number and can make those reads fail, so tests assert on reads without reaching into mock call arguments.

New archiver-sync cases: a pure re-time (count and rolling hash unchanged, timestamp and block hash updated, exactly one warning); a merge of two buckets into one with message indices preserved; a bucket spanning two co-timestamped blocks split by a reorg of the later one; a reorder rolled back by the bucket walk without reaching the per-message log queries; a head re-mined at the same number; rollover siblings re-downloaded whole when the rollback lands on their shared opening block; the watermark checking a below-finality bucket once and never again; a bucket that is never finalized being re-checked every pass; a spanning bucket waiting on its closing block to finalize; a bucket reorged out across a gap in syncing and detected even though it is now below finality; a failing L1 read warning and verifying nothing; the per-pass bound spreading the walk over several passes; and a catch-up over many buckets from a synced store using bulk events only. message_store gets the span accessors, the span shortening when a removal cuts a bucket, and the watermark being lowered both by a removal and by a re-delivery from another L1 block.

Both regressions were confirmed red before the fix: reverting the closing-block check fails the split case, and moving the walk back after the "L1 did not advance" guard fails the same-height re-mine.

Ran: full yarn build, yarn format, yarn lint archiver stdlib, archiver 597/597 (20 suites), stdlib messaging 57/57. No e2e.

Stacked on #25351

Fixes A-1880

…l back to the last canonical bucket

The archiver decides it is in sync with the Inbox by comparing message count
and consensus rolling hash. An L1 reorg that re-mines the same messages in a
different block leaves both unchanged while rewriting the bucket metadata built
from that block: its timestamp, sequence number and boundaries. Those feed the
censorship cutoff and the bucket hint a proposer publishes, so stale metadata
costs a slot.

Every sync pass now compares the L1 block hash each unverified bucket was opened
in against the live chain, walking up from a canonicality watermark the store
keeps. Buckets sharing an opening block cost a single read. On a mismatch the
archiver rolls back to the last canonical bucket and rewinds the message
syncpoint to just before its L1 block, so the normal fetch re-downloads the
reorged blocks; the rolling-hash rollback stays as the fallback.

Being at or below the finalized L1 block is not on its own a reason to skip a
bucket: a node that was not syncing when its block was reorged out holds an
orphaned block that has since been buried below finality. The watermark
therefore only advances over a bucket this node has itself compared against a
chain that had it finalized, and it is lowered whenever messages are removed.
…mically

Review follow-ups to the bucket canonicality check.

A bucket keyed on an L1 timestamp can absorb messages from a later block that
shares it, and a reorg of that later block alone splits the bucket while its
opening block stays canonical and the Inbox's total and rolling hash are
unchanged. Bucket snapshots now record the closing L1 block as well as the
opening one, both are compared against the live chain, and the watermark only
advances once the closing block is finalized. The snapshot type keeps this
internal: the check reads bucket spans rather than whole buckets.

The check also runs before the "L1 has not moved forward" guard, so a head
re-mined at the same number is caught instead of short-circuiting the whole
message sync until the chain advances.

Rolling back now resolves the target syncpoint before writing, then removes the
messages and moves the syncpoint in one transaction, so an interruption cannot
leave truncated messages behind a syncpoint that would never fetch them again.
The rolling-hash fallback shares that path. Re-delivering a stored bucket from a
different L1 block also lowers the watermark, so a bucket is never trusted on a
check made against blocks it no longer sits on.

Finally, the L1 reads are made in bounded parallel and capped per pass, so an
unfinalized tail cannot grow the per-tick read count without limit, and a read
that keeps failing is reported at warn rather than debug, since it blocks every
bucket above it from ever being verified.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant