bench(blocksync): per-block stage timing and a no-fsync switch for profiling

[PastaPastaPasta]

Block sync reports partset, verify, save and exec per block, but exec is a single number covering the two ABCI calls, our own durable writes, the mempool update and the event publish. That is most of a block, and it was opaque.

This adds two things, both off unless switched on.

TD_BLOCK_PERF=1 — stage timing

Times each stage of ApplyBlock and reports the means every 100 blocks. It also splits verify and records apply_gap, the time the applier spends waiting for the next block to arrive.

Two findings that came straight out of it:

The verify stage is a single BLS threshold verification. Splitting it:

verify_commit_sig=1.896 ms   verify_validate_block=0.009 ms

ValidateBlock is free; the whole stage is the signature. That is what #1427 and #1428 act on.

A single peer costs a third of the sync. apply_gap is 2.2–3.5 ms/block with one peer and 0.003 ms with two — the applier is waiting on block fetch, not executing:

peers	blocks/s	ms/block
1	97.1	10.30
2	140.5	7.12

Worth knowing when reading any block-sync benchmark, and worth telling operators.

TD_UNSAFE_NOSYNC=1 — drop the fsync from store writes

This is a benchmarking correction rather than a tuning knob.

Go's os.File.Sync() on darwin issues F_FULLFSYNC, not fsync. Measured on an M-series Mac:

go File.Sync mean ms: 6.776
plain fsync  mean ms: 0.231

Tenderdash does three synchronous writes per block during block sync — BlockStore.SaveBlock, saveABCIResponses, state.Save — so on macOS it spends about 20 ms a block that a Linux node does not. Same binaries, same chain:

	blocks/s	verify	save	exec
fsync	34.6	4.20	5.42	19.55
TD_UNSAFE_NOSYNC=1	92.4	3.47	0.04	6.79

fb_save_abci_resp 6.60 → 0.03 ms, fb_save_state 6.03 → 0.05 ms. Leaving it on hides everything else in a profile taken on macOS.

The name is deliberate. It removes durability: a power loss can leave the block store behind the application, which the replayer rejects with ErrAppBlockHeightTooHigh. Never set it on a node whose data matters.

How this was tested

Used throughout a full mainnet Platform replay, genesis to height 424,981, and for every A/B measurement behind #1427 and #1428.

internal/state/... and internal/blocksync/... pass.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7faf8567-c2ed-4dc2-a2bc-536a4cba49a2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thepastaclaw]

🕓 Ready for review — 22 ahead in queue (commit aefda6c)
Queue position: 23/39 · 2 reviews active
ETA: start ~09:31 UTC · complete ~10:36 UTC (median 1h 4m across 30 recent reviews; 2 slots)
Queued 5h 22m ago · Last checked: 2026-08-31 21:40 UTC