refactor(bbapi): migrate bb + bb.js to ipc-codegen/ipc-runtime; delete legacy bb::ipc - #25362
Open
charlielye wants to merge 1 commit into
Open
refactor(bbapi): migrate bb + bb.js to ipc-codegen/ipc-runtime; delete legacy bb::ipc#25362charlielye wants to merge 1 commit into
charlielye wants to merge 1 commit into
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
August 29, 2026 17:17
fb6fad4 to
edcb954
Compare
charlielye
marked this pull request as ready for review
August 29, 2026 17:17
This was referenced Aug 29, 2026
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
5 times, most recently
from
August 29, 2026 18:53
4f31e79 to
ccda574
Compare
…e legacy bb::ipc Combined redo of #23612 + #23614 + #23613. Those predate envelope ids (the 8-byte request-id frame prefix), which broke wire compatibility between the old and new stacks, so bb, bb.js and bb-rs have to move together. bb now serves msgpack through ipc-runtime on every transport: `bb msgpack run` maps ""/"-" to a stdio pipe server, .sock to UDS and .shm to shared memory, all dispatching through one generated handler with envelope framing (a plain file keeps the offline bare-frame replay). The command surface comes from a checked-in bb_schema.json in the friendly ipc-codegen dialect, with handlers implemented as thin adapters over the existing domain commands. The typed cbind entrypoint is replaced by the ipc-codegen FFI contract ipc_ffi_entry, so wasm and static-linking consumers speak the same payload as the transports. That makes the legacy machinery consumer-free, so it goes: the in-tree ipc library and its benchmark, the nodejs_module msgpack client (bb.js now uses ipc-runtime's NAPI addon, and nodejs_module only exports LMDBStore), the Command/CommandResponse named unions with their execute dispatcher, and the func_traits/schema reflection behind the CBIND macros. Both client libraries now generate from ipc-codegen rather than forks of it. bb.js/src/cbind (a stale copy of the generator: schema visitor, TS and rust backends, naming, friendly-schema lowering) is deleted in favour of calling ipc-codegen, and barretenberg-rs drops its hand-written Backend trait, error type, Fr/Point types and both backends — codegen emits those, and ipc_runtime::IpcClient plugs in as the Backend, so the crate is ~89% generated with a deprecated BarretenbergApi shim keeping the published surface. The hand-rolled PipeBackend goes with them; its tests now run over ipc-runtime's UDS transport. ipc-codegen gains --strip-type-prefix (bb.js and barretenberg-rs publish unprefixed names while wire tags keep the service prefix) and three rust serde fixes: fq[2] pairs, [u8; N] and above-cutoff byte arrays were encoded as sequences of integers rather than msgpack bin, which the C++ side rejects. ipc-runtime gains the pipe transport (fd pair or stdio, peer EOF requests shutdown), a sync spawned-process backend for callers that cannot await, pre-spawn removal of stale shm segments (they are created O_EXCL, so a killed server's leftovers blocked the next one), and an unref option so a spawned backend cannot hold the Node event loop open while idle. Its rust and zig build files now discover the C++ sources instead of keeping hand-written copies of the CMake list, which had silently drifted. Also fixes two latent breakages from the ts/ -> ts/bb.js/ restructure that blocked wasm and pinned-flow tests: the barretenberg_wasm symlinks and the chonk pinned-inputs repo-root resolution were both one directory short.
charlielye
force-pushed
the
cl/ipc-bb-bbjs-migrate
branch
from
August 29, 2026 19:12
ccda574 to
fa04e10
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Combined redo of #23612 + #23614 off
next. Those PRs predate envelope ids (the 8-byte request-id frame prefix), which broke wire compatibility between old and new stacks — bb, bb.js, and bb-rs must move together, so this is one PR. The old PRs were used as the recipe; they'll be closed pointing here.bb (C++)
bbapi/bb_schema.jsonchecked in, in the friendly ipc-codegen dialect (same as avm/cdb/wsdb). CMake runs ipc-codegen at build time (client+server,bb::bbapinamespace); the schema is also embedded sobb msgpack schemaprints it.bbapi_handlers.*implement the generated dispatch (Responder-style, exactly-once ok/error), as thin adapters over the existing domainCmd{...}.execute(ctx)commands. All 64 commands.bb msgpack runserves every transport through ipc-runtime —-/empty → stdio pipe server,.sock→ UDS,.shm→ shared memory — one generated handler, envelope framing everywhere. A plain-file input keeps the offline bare-frame replay for wasm/debug flows.bbapi()replaced by the ipc-codegen FFI contractipc_ffi_entry(input, len, out, out_len)— same msgpack payload as the transports, no envelope, output is free()-compatible.barretenberg/ipc/+ipc_bench,nodejs_module/msgpack_client/(nodejs_module now only exports LMDBStore),bbapi_execute.*(Command/CommandResponse named unions + dispatcher), func_traits/schema reflection + CBIND macros + msgpack_schema test.ipc-runtime
PipeServer/PipeClientover an fd pair or stdio, identical envelope framing to UDS, peer EOF → shutdown. Used bybb msgpack runon stdio and by bb-rs.unrefoption onSpawnedProcessBackend(child + idle-socket unref, re-ref while calls are in flight) so bb.js callers that never destroy() can't hang node.bb.js
bb_schema.json(vendored friendly-schema lowering), runs on plainnode --experimental-strip-types(ts-node dropped), and strips theBbservice prefix from identifiers so the public API surface is unchanged while wire tags carry the prefix.SpawnedProcessBackend; SHM sync/async backends use ipc-runtime's NAPI clients (request-id correlation replaces FIFO queues); wasm backends call theipc_ffi_entryexport. Unused stdio pipe backend deleted.barretenberg-rsgenerated sources) regenerates from the same pipeline;Shutdownis gone (pipe EOF replaces it — it was the only sender).bb-rs
ipc_ffi_entry.fq[2](Fq2) fields now getserde_array2_bytes, so G2 coordinates serialize as bin32 rather than integer arrays.Schema dialect notes (vs the raw dump the old PRs used)
Shutdownremoved;CircuitKindis a plainu8on the wire (the alias collided with the domain enum atbb::bbapiscope); Fq2/G2 coords arefq[2]; the shared vk-data struct isVkData(prefix-stripping made the old name collide withCircuitComputeVkResponse).Drive-by fixes
Two latent breakages from the
ts/→ts/bb.js/restructure, both one directory short, both blocking local validation of this change (verified untouched by this branch onnext):src/barretenberg_wasm/*.wasm.gzsymlinks resolved tobarretenberg/ts/cpp/..., so any src-based wasm test failed with ENOENT.chonk_pinned_inputs.test.ts'sfindRepoRoot()producedbarretenberg/barretenberg/cpp/chonk-pinned-flows.Validation
bbapi_testsgreen (the knownChonkPinnedIvcInputsTestfixture flake passed on rerun);c_bindexception tests green throughipc_ffi_entry.ipc_ffi_entrypath returns the identical hash. (In wasm a failing command aborts via thethrow_or_aborthost import rather than returning an error response — unchanged behavior, since thewasmpreset definesBB_NO_EXCEPTIONSand the old cbind guarded its catch the same way.)libbb-external.a.