fix: store LongBits words unsigned

[SgtPooki]

LongBits stores its low word signed (this.lo = lo | 0). For a 64-bit value with hi == 0 and lo in [2^31, 2^32) the word is negative, so writeVarint64 writes one byte into the 5 bytes length() reserved. The other 4 bytes are whatever was in the allocUnsafe slab.

• affected: uint64, int64
• not affected: sint64, fixed64, sfixed64

Repro

# protons-runtime 8.0.1 (and 5.0.4 through main)
node --zero-fill-buffers --input-type=module -e 'import {writer} from "protons-runtime"; const w = writer(); w.uint64(2147483648n); console.log(Buffer.from(w.finish()).toString("hex"))'
# 0000000000

# protobufjs 8.8.1, same value
node -e 'const {Writer} = require("protobufjs/minimal"); console.log(Buffer.from(Writer.create().uint64(2147483648).finish()).toString("hex"))'
# 8080808008

# protons-runtime 5.0.3 (last release wrapping protobufjs), same command as the first
# 8080808008

Without --zero-fill-buffers the 4 bytes are Buffer pool leftovers, so the same node encodes differently depending on what ran before it. Browser builds of uint8arrays zero-fill, so there it is always the 0000000000 form.

# same UnixFS node, 200 times, with some allocations in between
node --input-type=module -e 'import {UnixFS} from "ipfs-unixfs"; import * as dagPB from "@ipld/dag-pb"; import {sha256} from "multiformats/hashes/sha2"; import {CID} from "multiformats/cid"; const cids = new Set(); for (let i = 0; i < 200; i++) { for (let j = 0; j < 50; j++) Buffer.allocUnsafe(1 + (i * 7 + j * 13) % 500).fill(i + j); const bytes = dagPB.encode({Data: new UnixFS({type:"file",blockSizes:[2147483648n]}).marshal(), Links: []}); cids.add(CID.createV1(dagPB.code, await sha256.digest(bytes)).toString()) } console.log(cids.size, "distinct CIDs for the same node")'
# 25 distinct CIDs for the same node

Impact

UnixFS filesize and blocksizes are uint64. A file node is invalid protobuf when its filesize or any blocksizes entry is in [2^31, 2^32). That is every multi-block file of 2 GiB to 4 GiB, plus larger files with a subtree in that range.

• the protons reader skips the bad tags, so JS round trips pass
• Go fails with proto: cannot parse invalid wire-format data on the zero form, or decodes wrong values (filesize 0) when the leftovers happen to parse
• on Node the root CID varies between runs
• end-to-end symptom: add of a 2 to 4 GiB file produces a UnixFS root Kubo cannot parse (protons-runtime uint64 bug) filecoin-project/filecoin-pin#773

Where it came from

• d101804 (fix: port protobuf reader/writer to ts #60, "fix: port protobuf reader/writer to ts", 2023-10-13), released as protons-runtime 5.0.4 the same day
• protobufjs stores both words with >>> 0 (src/util/longbits.js#L23-L29 at protobufjs-v7.2.5, the line 5.0.3 wrapped); the port changed them to | 0
• protons-runtime: 5.0.3 and earlier correct, 5.0.4 through 8.0.1 and main broken (bisected, then each major checked from npm)
• ipfs-unixfs: 11.0.0 through 11.1.3 (^5.0.0) break on any install after 2023-10-13; 11.1.4 and later (^5.4.0, ^5.5.0, ^6.0.1, ^7.0.0, ^7.1.1, ^8.0.1) have never resolved to a working runtime; a lockfile pinned to 5.0.3 is fine

Change

Restore >>> 0 on both words. Regression test at the window edges for uint64 and int64, red on main, green here.

Checked 13,483 encodings across all five 64-bit codecs before and after: the 520 in-window cases change to the correct bytes, nothing else moves. INT64_MIN is mis-encoded on int64/sint64/sfixed64 both before and after; separate overflow in fromBigInt, not touched here.

fixes #213

[SgtPooki]

@wemeetagain @achingbrain can we get this in ASAP? two-line fix with test included. every 2 to 4 GiB file written through ipfs-unixfs since 2023-10-13 has a root node Go can't read.

[SgtPooki]

other callers of uint64 in the libp2p tree that hit the same window (read from the generated codecs, not run against go):

• ipns ttl is nanoseconds, so any ttl between 2.15s and 4.29s encodes corrupt. helia's default is 1h so nobody has hit it, but a short ttl would produce a record go-ipns can't parse.
• @libp2p/circuit-relay-v2 reservation expire is unix seconds. values cross 2^31 on 2038-01-19, after which every js relay reservation is malformed until this lands. same for the data limit if anyone sets it to 2 to 4 GiB.

so this isn't only a unixfs problem.

[hugomrdias]

LGTM

[achingbrain]

Looks like publishing to npm failed here.