Skip to content

Repair missing UTXO address rows on startup - #464

Merged
j0ntz merged 3 commits into
masterfrom
jon/ltc-missing-data-layer-address
Oct 9, 2026
Merged

j0ntz merged 3 commits into
masterfrom
jon/ltc-missing-data-layer-address

Conversation

@j0ntz

@j0ntz j0ntz commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

CHANGELOG

Does this branch warrant an entry to the CHANGELOG?

  • Yes
  • No

Dependencies

none

Description

A Litecoin wallet on Android (4.52.0 staging) showed Engine Failed: Missing data-layer address with 'bip49/0/0' path during initialization on every login. The device log has the same failure in four consecutive logins. Each login builds a new engine, which puts the bad state on disk.

Cause. The wallet's path index still counts its addresses, but the row for an address cannot be read. Either bucket file can produce this: tables/addressByScriptPubkey/a914.json (every bip49 address of the wallet shares it) or tables/scriptPubkeyByPath/bip49_0/0.json. The storage layer treats an empty or unparseable bucket as "row not found", and initializeAddressSubscriptions threw on the first missing row. The log does not prove what emptied the file on the reporter's device. The leading candidate is a write cut short by a process kill: Android's disklet write truncates the file before writing it.

Fix.

  • initializeAddressSubscriptions re-derives a missing row from the wallet keys and its path, saves it, and continues. It still throws if the row cannot be read back after the save.
  • run() clears its running flag when initialization throws. A later call then retries instead of skipping initialization.

A repaired row starts with default metadata (unused, never queried). The engine queries it again. Balances are keyed by script, which keeps a repaired address from being counted twice.

Testing.

  • New AddressRepair.spec.ts runs a real Litecoin engine over a memory disklet. It empties each bucket file, restarts the engine, and expects the file content back. A third test covers the retry. All three fail on master with the production error and stack.
  • iOS simulator, agent test account, empty Litecoin wallet: emptied a914.json in the app container. The published 3.15.0 bundle shows the Engine Failed row from the report. With this branch's bundle the wallet loads and Receive shows its address. The file has the same sha1 as the backup afterward. Emptying the path bucket heals the same way.
  • The first commit makes the live-network Blockbook tests opt-in: LIVE_NETWORK_TESTS=1 npm test runs them. They connect to btc-wusa1.edge.app, and on 2026-10-08 two of them (fetchAddress, fetchAddressUtxos) timed out on this branch and on master, which fails npm run verify for any PR. The commit touches only that spec file and can be dropped on its own.

Test evidence

4670089
Retry UTXO engine init after a failure
2026-10-08

1. before engine failed

2. after wallet loads

3. after receive

Note

Medium Risk
Touches UTXO engine startup and on-disk address recovery; incorrect repair could affect sync state, though repaired rows re-sync and balances stay keyed by scriptPubkey.

Overview
Fixes UTXO wallets that could not load with "Missing data-layer address" when on-disk address buckets were empty or corrupt (e.g. after a truncated write) while path counts still expected those rows.

During initializeAddressSubscriptions, a missing row is now re-derived from wallet keys and the derivation path, saved via the data layer, and used to continue startup; it still fails if the row cannot be read back after save. run() clears its running guard when initialization throws so a later addGapLimitAddresses / start can retry instead of being skipped permanently.

Adds AddressRepair.spec.ts (Litecoin engine, corrupted bucket files, init retry). Live Blockbook integration tests are opt-in via LIVE_NETWORK_TESTS=1 so default CI does not hit external servers.

Reviewed by Cursor Bugbot for commit b9f1431. Bugbot is set up for automated code reviews on this repo. Configure here.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@j0ntz
j0ntz force-pushed the jon/ltc-missing-data-layer-address branch from 70f5d4d to 4670089 Compare October 8, 2026 17:59
j0ntz added 3 commits October 9, 2026 12:29
The Blockbook spec connects to a live Edge server. When that server is
slow, the suite fails for every branch. The live block now runs only
with LIVE_NETWORK_TESTS=1.
The address tables can report an address count while the rows behind
it are unreadable. Android writes a file by truncating it and then
writing the new contents, so a process that dies in between leaves an
empty bucket file, and the storage layer reads an unparseable bucket
as a missing row. Every address of one script type shares a bucket,
so one cut-short write loses all of them and the engine failed with
"Missing data-layer address" on every later launch.

Each row derives from the wallet keys and its path, so rebuild the
missing row during initialization and let the address sync again.
A failed initialization left the processor marked as running, so the
next start skipped it and connected to servers with no addresses to
subscribe to. Clear the flag when initialization throws so the next
call runs it again.
@j0ntz
j0ntz force-pushed the jon/ltc-missing-data-layer-address branch from 4670089 to b9f1431 Compare October 9, 2026 19:32
@j0ntz
j0ntz merged commit 7ebd101 into master Oct 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants