DEV.md

Developer guide

The architecture, protocol details, and every load-bearing design fact are in the notebooks (nbs/00_core.ipynb is source, docs, and tests at once; read it top to bottom). This file holds what a contributor needs beyond that: the reference-implementation context the design came from, decisions that would otherwise need re-litigating, and mechanics.

What the reference implementations taught us

jupygate replaces the zmq-to-websocket bridge that jupyter_server implements in jupyter_server/services/kernels/connection/channels.py (~900 lines) and that kernel_gateway merely wraps with auth/CORS mixins. Reading that file explains most of jupygate's structure by contrast:

• The "nudge" (channels.py, nudge()): jupyter_server opens fresh zmq streams per websocket connection, so every connect races zmq's PUB/SUB slow-joiner problem and must retry kernel_info_request on transient sockets until iopub proves live, with abort paths for busy kernels, mid-nudge restarts, and closed sockets. jupygate's one-persistent-channel-set-per-kernel design makes all of that once-per-kernel, and JEP 65 welcomes (ipymini, modern ipykernel) reduce it to a bounded wait. The fallback loop for non-welcome kernels is jupyter_client's wait_for_ready semantics, unified into the same code path: welcome or any other iopub traffic both prove the subscription.
• Reconnect buffering (start_buffering/get_buffer in kernelmanager.py): jupyter_server buffers per-session output when the last websocket drops and replays on reconnect, via a handoff of the per-connection zmq streams - an unbounded list, one buffered session per kernel, destroyed when a different session connects, no TTL. jupygate's version (the notebook's Reconnects section) is simpler and stricter because the zmq side never moves: a disconnected client's ClientQueue stays behind as a bounded ring keyed by session id, replayed in order into the same-session reconnect, with a synthesized drop warning and current status when the ring overflowed. Policies: only client-supplied session ids are buffered, the ring caps everything while detached (attached queues bound iopub only and never drop status), and a reaper collects rings older than buffer_secs (default 3600; 0 disables). jupyasyncclient holds up the client half: it redials with the same session id, resends the frame that died mid-send, keeps pending reply futures alive across the drop, and gives up fast when an HTTP probe says the kernel is gone.
• iopub rate limiting (_limit_rate): protects browsers from output floods, with stderr warnings and window bookkeeping. jupygate instead bounds each client queue and never drops status (same policy as ipymini's own IOPub thread), which keeps the busy/idle picture truthful and makes the drop policy explicit and testable rather than libzmq's silent per-subscriber HWM behavior.
• Session identity: kernels address input_request on their stdin ROUTER to the zmq identity of the shell request that called input(); jupyter_client and jupyter_server make that work by giving shell and stdin DEALERs the same identity. jupygate does the same with the gateway session's identity, and because all clients share it, the mux stamps remembered input_request headers onto parentless input_replys (most clients never set that parent).

Decisions that look open but aren't

• Embedders bypass HTTP entirely. The Kernels registry plus mux.add with an in-process send callback is the supported embedding surface (the notebook's registry demo shows it end to end); an MCP server or test harness reusing these internals should never need starlette. The HTTP app is routes over the registry, plus auth.
• Kernel identity is ids only. No keys, no labels, no get-or-create in the API. Keyed reuse is client-side sugar (see jupyasyncclient's ensure_kernel); gateway-side keys only pay off when uncoordinated clients race for the same kernel, which nothing we run does. A key parameter is addable compatibly later.
• Legacy websocket protocol only, for now. The v1.kernel.websocket.jupyter.org subprotocol frames the raw zmq parts (an offset header, the channel name, then the signed message parts verbatim), so the gateway would forward kernel traffic without deserializing and re-encoding JSON; jupyter_server negotiates it via websocket subprotocols and falls back to legacy. jupygate speaks only the legacy JSON-plus-channel-key protocol because that is what existing web clients (and jupyasyncclient) already speak. Costs to be aware of before reaching for v1: each iopub message is decoded once (Session.deserialize in the pump, which also verifies the kernel's HMAC) and encoded once for all clients (to_frame in the pump; queues hold encoded frames). Encoding in the pump has a consequence the reconnect work paid for once already: the pump is the only coroutine draining the SUB socket, so per-message cost there backs the socket up during floods, which is why the SUB runs RCVHWM=0 and the pump yields per message. v1 pays by removing the decode/encode entirely, at the price of implementing subprotocol negotiation and a second codec on both gateway and client. How to check whether any of this is needed: run an output-flood workload (the tests/ flood test scaled up, or a cell printing tens of MB) with a realistic client count, and profile the gateway process (py-spy top --pid <gateway>, or cProfile around serve). If json.dumps/loads and Session.deserialize dominate gateway CPU while clients lag, and the gateway rather than the kernel or the network is the bottleneck, v1 is justified (~30 lines each side; reference implementations in jupyter_server/services/kernels/connection/base.py:80-115).
• starlette + uvicorn. The service returns JSON and frames, so fasthtml's rendering layer would be unused weight; starlette 1.0 (2026, post-encode, under the Kludex org) is the stable API written against - note lifespan=, since on_startup/on_shutdown were removed in 1.0. The real commitment is ASGI: if the fan-out path ever profiles as server-bound, granian serves the same app unchanged.
• The CLI's reloader is always on, watching one touch file. Every jupygate run watches ~/.local/state/jupygate/reload/ (created at startup): touching r.py there restarts the whole gateway, killing all kernels - and that is the point: it is the deliberate restart-with-new-code lever after upgrading jupygate or a kernel package in the venv, needing no pid hunting (--reload additionally watches the package source; timeout_graceful_shutdown=5 bounds the websocket drain so a restart cannot hang). serve() and embedders never reload.
• HTTP interrupt is SIGINT; in-band interrupt_request also passes through the mux. Both are tested; the signal path covers kernels configured for signal interrupts and clients without a control channel.
• Creation takes explicit argv/env/cwd/username. No kernelspec lookup: clients are trusted or token-authed, which is why kernel_gateway's env-whitelisting machinery has no equivalent here. With auth off or a leaked token this API is arbitrary code execution by design - like any Jupyter kernel server.

zmq boundedness

No zmq call the gateway makes may block without bound: zmq fails by blocking or silently dropping, and a blocked coroutine or a wedged term looks like a dead gateway. How each site satisfies it: every socket is linger=0, and KernelChannels.close closes all sockets before ctx.term(), so term returns promptly; KernelChannels.send awaits the socket send, so a kernel that stops reading applies backpressure to that kernel's pump (bounded by kernel lifecycle: deleting the kernel closes the sockets and fails the pending send) instead of accumulating unsent messages invisibly inside pyzmq; the iopub SUB never sends, and it receives with RCVHWM=0, trading bounded-by-HWM memory for losslessness: libzmq HWM drops are silent and status-blind, and the kernel's own iopub queue policy bounds what a flood can push at us. A new zmq call site must state how it satisfies this invariant.

Welcome adoption elsewhere

ipymini >= 0.1.17 binds iopub as XPUB with XPUB_VERBOSE (one welcome per subscriber; plain XPUB dedups per topic, which is why modern ipykernel's second subscriber gets no welcome). Downstream consumers that can delete compensations once they adopt the unified ready-wait: conkernelclient's start_channels (retry loop, iopub drain, 0.2s settling sleep) and reconnect; ipymini's own test harness. jupyasyncclient needs nothing (no zmq; websockets have no slow-joiner problem). true_async_client keeps its hand-rolled nudge; it is being superseded by the websocket path.

Tests and mechanics

The notebooks are the primary tests (nbdev-test nbs/00_core.ipynb runs the whole story against live kernels); tests/ holds only what makes a bad docs page: load, races, and edge cases (two-client stdin stamping, floods, concurrent routing, both interrupt paths, kernel death, replay-guard interaction with ipymini's HMAC dedup, large buffers, lifespan reap, mid-flood reconnect, buffering policy and TTL). pytest -q runs them with pytest-timeout bounds (timeout_method = thread, because a hang inside a C call like ctx.term() survives signal-based timeouts). Style is fastai (chkstyle jupygate/core.py); the .py files are generated - edit the notebooks. JUPYGATE_TEST_URL points the suite at an external gateway instead of the in-thread server, turning it into a conformance suite for any implementation of this API (two tests that inspect in-process internals skip themselves). Kernel creation still uses this venv's ipymini, so the external gateway must run on the same machine.

CI needs ipymini >= 0.1.17 on PyPI (the welcome release), so the release order for first push is: ipymini, then jupygate, then jupyasyncclient (whose notebooks exercise this gateway).