Skip to content

refactor(bbapi): migrate bb + bb.js to ipc-codegen/ipc-runtime; delete legacy bb::ipc - #25362

Open
charlielye wants to merge 1 commit into
nextfrom
cl/ipc-bb-bbjs-migrate
Open

refactor(bbapi): migrate bb + bb.js to ipc-codegen/ipc-runtime; delete legacy bb::ipc#25362
charlielye wants to merge 1 commit into
nextfrom
cl/ipc-bb-bbjs-migrate

Conversation

@charlielye

@charlielye charlielye commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Combined redo of #23612 + #23614 off next. Those PRs predate envelope ids (the 8-byte request-id frame prefix), which broke wire compatibility between old and new stacks — bb, bb.js, and bb-rs must move together, so this is one PR. The old PRs were used as the recipe; they'll be closed pointing here.

bb (C++)

  • Schema-first bbapi: bbapi/bb_schema.json checked in, in the friendly ipc-codegen dialect (same as avm/cdb/wsdb). CMake runs ipc-codegen at build time (client+server, bb::bbapi namespace); the schema is also embedded so bb msgpack schema prints it.
  • Handlers: bbapi_handlers.* implement the generated dispatch (Responder-style, exactly-once ok/error), as thin adapters over the existing domain Cmd{...}.execute(ctx) commands. All 64 commands.
  • Serve: bb msgpack run serves every transport through ipc-runtime — -/empty → stdio pipe server, .sock → UDS, .shm → shared memory — one generated handler, envelope framing everywhere. A plain-file input keeps the offline bare-frame replay for wasm/debug flows.
  • FFI/wasm: typed-cbind bbapi() replaced by the ipc-codegen FFI contract ipc_ffi_entry(input, len, out, out_len) — same msgpack payload as the transports, no envelope, output is free()-compatible.
  • Deletions (consumer-free after the above): legacy barretenberg/ipc/ + ipc_bench, nodejs_module/msgpack_client/ (nodejs_module now only exports LMDBStore), bbapi_execute.* (Command/CommandResponse named unions + dispatcher), func_traits/schema reflection + CBIND macros + msgpack_schema test.

ipc-runtime

  • New pipe transport (C++): PipeServer/PipeClient over an fd pair or stdio, identical envelope framing to UDS, peer EOF → shutdown. Used by bb msgpack run on stdio and by bb-rs.
  • TS: unref option on SpawnedProcessBackend (child + idle-socket unref, re-ref while calls are in flight) so bb.js callers that never destroy() can't hang node.

bb.js

  • Generator consumes the checked-in bb_schema.json (vendored friendly-schema lowering), runs on plain node --experimental-strip-types (ts-node dropped), and strips the Bb service prefix from identifiers so the public API surface is unchanged while wire tags carry the prefix.
  • Backends: UDS backend is a thin wrapper over SpawnedProcessBackend; SHM sync/async backends use ipc-runtime's NAPI clients (request-id correlation replaces FIFO queues); wasm backends call the ipc_ffi_entry export. Unused stdio pipe backend deleted.
  • Rust client (barretenberg-rs generated sources) regenerates from the same pipeline; Shutdown is gone (pipe EOF replaces it — it was the only sender).

bb-rs

  • Pipe backend speaks the envelope framing (serial ids, echoed-id verification); FFI backend binds ipc_ffi_entry.
  • Fixed a latent generator gap surfaced by the friendly schema: fq[2] (Fq2) fields now get serde_array2_bytes, so G2 coordinates serialize as bin32 rather than integer arrays.

Schema dialect notes (vs the raw dump the old PRs used)

  • Shutdown removed; CircuitKind is a plain u8 on the wire (the alias collided with the domain enum at bb::bbapi scope); Fq2/G2 coords are fq[2]; the shared vk-data struct is VkData (prefix-stripping made the old name collide with CircuitComputeVkResponse).

Drive-by fixes

Two latent breakages from the ts/ts/bb.js/ restructure, both one directory short, both blocking local validation of this change (verified untouched by this branch on next):

  • src/barretenberg_wasm/*.wasm.gz symlinks resolved to barretenberg/ts/cpp/..., so any src-based wasm test failed with ENOENT.
  • chonk_pinned_inputs.test.ts's findRepoRoot() produced barretenberg/barretenberg/cpp/chonk-pinned-flows.

Validation

  • ipc-runtime: C++ tests incl. 6 new pipe tests, + ASAN; TS tests 18/18.
  • bb: bbapi_tests green (the known ChonkPinnedIvcInputsTest fixture flake passed on rerun); c_bind exception tests green through ipc_ffi_entry.
  • bb.js full suite 10/10 suites, 50 passed / 0 failed, exercising every transport — wasm, native socket, socket pipelined, SHM async, SHM pipelined, SHM sync (the SHM legs cover the NAPI clients' request-id correlation under pipelining).
  • Pinned Chonk IVC flows prove and verify through bb.js on both the native and wasm backends.
  • Live UDS smoke against real bb: pipelined poseidon2, correct hashes, clean SIGTERM. The wasm ipc_ffi_entry path returns the identical hash. (In wasm a failing command aborts via the throw_or_abort host import rather than returning an error response — unchanged behavior, since the wasm preset defines BB_NO_EXCEPTIONS and the old cbind guarded its catch the same way.)
  • bb-rs: pipe leg 14/14 against the new bb binary; FFI leg 75/75 against libbb-external.a.

@socket-security

socket-security Bot commented Aug 29, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​@​aztec/​ipc-runtime@​0.0.0-use.local100100100100100

View full report

…e legacy bb::ipc

Combined redo of #23612 + #23614 + #23613. Those predate envelope ids (the
8-byte request-id frame prefix), which broke wire compatibility between the
old and new stacks, so bb, bb.js and bb-rs have to move together.

bb now serves msgpack through ipc-runtime on every transport: `bb msgpack run`
maps ""/"-" to a stdio pipe server, .sock to UDS and .shm to shared memory,
all dispatching through one generated handler with envelope framing (a plain
file keeps the offline bare-frame replay). The command surface comes from a
checked-in bb_schema.json in the friendly ipc-codegen dialect, with handlers
implemented as thin adapters over the existing domain commands. The typed
cbind entrypoint is replaced by the ipc-codegen FFI contract ipc_ffi_entry,
so wasm and static-linking consumers speak the same payload as the transports.

That makes the legacy machinery consumer-free, so it goes: the in-tree ipc
library and its benchmark, the nodejs_module msgpack client (bb.js now uses
ipc-runtime's NAPI addon, and nodejs_module only exports LMDBStore), the
Command/CommandResponse named unions with their execute dispatcher, and the
func_traits/schema reflection behind the CBIND macros.

Both client libraries now generate from ipc-codegen rather than forks of it.
bb.js/src/cbind (a stale copy of the generator: schema visitor, TS and rust
backends, naming, friendly-schema lowering) is deleted in favour of calling
ipc-codegen, and barretenberg-rs drops its hand-written Backend trait, error
type, Fr/Point types and both backends — codegen emits those, and
ipc_runtime::IpcClient plugs in as the Backend, so the crate is ~89% generated
with a deprecated BarretenbergApi shim keeping the published surface. The
hand-rolled PipeBackend goes with them; its tests now run over ipc-runtime's
UDS transport.

ipc-codegen gains --strip-type-prefix (bb.js and barretenberg-rs publish
unprefixed names while wire tags keep the service prefix) and three rust
serde fixes: fq[2] pairs, [u8; N] and above-cutoff byte arrays were encoded as
sequences of integers rather than msgpack bin, which the C++ side rejects.

ipc-runtime gains the pipe transport (fd pair or stdio, peer EOF requests
shutdown), a sync spawned-process backend for callers that cannot await,
pre-spawn removal of stale shm segments (they are created O_EXCL, so a
killed server's leftovers blocked the next one), and an unref option so a
spawned backend cannot hold the Node event loop open while idle. Its rust and
zig build files now discover the C++ sources instead of keeping hand-written
copies of the CMake list, which had silently drifted.

Also fixes two latent breakages from the ts/ -> ts/bb.js/ restructure that
blocked wasm and pinned-flow tests: the barretenberg_wasm symlinks and the
chonk pinned-inputs repo-root resolution were both one directory short.
@charlielye
charlielye force-pushed the cl/ipc-bb-bbjs-migrate branch from ccda574 to fa04e10 Compare August 29, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant