Request for Comments: Modular AirStack — module repos, slot contracts, marketplace, and distributed CI/docs #379
andrewjong
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
RFC: Modular AirStack — module repos, slot contracts, a module marketplace, and distributed CI/docs
Output of a design session on making AirStack modular rather than monolithic. Written to be self-contained: a reader (human or AI agent) should be able to implement from this document alone. Repo facts current as of
develop@55d9b887(VERSION0.19.0-alpha.9).Scope: this RFC covers how modules are packaged, integrated, tested, distributed, and documented. Its companion, RFC #380: Heterogeneous AirStack, builds on this machinery to configure heterogeneous deployments — vehicle variants, mixed multi-robot fleets, top-level configuration, and (future) cross-embodiment platforms.
1. Problem statement
Projects branched from AirStack build features we want as optional trunk capabilities, but today the only paths are merging into the monolith or living in a fork. Motivating cases: DFM2's Isaac Sim disturbance library (buried in a whole-repo fork), an OptiTrack motion-capture state-estimation integration with its own tests, and a custom global planner.
Pain points to solve:
local.launch.xml,local_droan_cpu.launch.xml,local_macvo_obstacle_avoidance.launch.xmlare near-copies with alternate wirings commented out).Design principles (decided)
2. Module anatomy: a thin repo with a manifest
A module repo contains only its packages plus
module.yaml:Mapping the motivating cases: OptiTrack →
ros_package,slot: state_estimator,provides: [odometry]; custom planner →slot: global_planner; DFM2 →isaac_extension(Isaac supports extension search paths — lowest-coupling pilot).3. Sync and launch integration
Sync:
modules.repos(vcstool format — not submodules).airstack module add <name|url>clones into a gitignoredmodules/dir; overlay plumbing places each module bytype/targets: ROS packages symlinked intorobot/ros_ws/src/modules/(colcon picks them up), Isaac extensions onto the extension path, compose fragments merged viaCOMPOSE_FILE. Trunk never carries module code. CLI surface:airstack module add|remove|list|sync|doctor|create|search|extract,airstack init --release <X>,module create --in-tree/doctor --drift(§11).Slots: the layers' implicit roles become named slots (
state_estimator,local_planner,local_world_model,global_planner,global_world_model,controller,behavior). Each slot's contract = the standard topics (already indocs/robot/autonomy/integration_checklist.md) as required launch args, plus QoS profiles and TF frame/units conventions — both are classic silent failures (best-effort-vs-reliable mismatch receives nothing; ENU vs NED) thatdoctormust check. Every module — trunk's included — ships a launch file honoring its slot's arg interface.stack.yamlreplaces hand-edited bringup XML. Trunk's default stack, expressed as data (module names per the reference implementations):Stacks compose by inheritance, so experiments are diffs, not copies — here the VLA stack from #380's fleet example, exercising §4's
replacessemantics:A generic Python bringup reads the resolved stack, includes each module's launch file, passes contract topics. The three
local_*.launch.xmlvariants collapse into preset stacks like these. (Trunk precedent for YAML-driven collection:tests/colcon_unit_test_packages.yaml.)Where overrides land: each
<module>.<key>resolves against the module's declaredinterface:(§2). A key matching a launch arg is passed straight through to the module's launch file; if the arg is annotatedrole: input|outputit's a topic endpoint — the resolver auto-wires unset ones by finding a provider, and an explicit override pins the choice (droan.disparity_inexists because two disparity sources are available). Therole/rungannotations are also what conflict detection and #380's split-placement checks operate on — a split may only cut topic-endpoint args. Any other key must exist in the module'sparamsfile and becomes a ROS 2 parameter override layered onto its defaults (bringup emits a merged per-robot param file, extending the config precedence chain into params). A key matching neither is adoctorerror: overrides are typo-checked against the manifest, never silently ignored. How stacks are assigned per robot in a multi-robot deployment — and the top-level config that selects them — is #380's territory; a stack.yaml itself stays a single-robot artifact.Wiring resolution — modules agree with rungs, never with each other. Pairwise agreement (MACVO must know DROAN's topic names) is O(N²) and is what today's hand-maintained remap blocks encode. Instead, in priority order:
odometry_out: {role: output, rung: odometry}, DROAN declaresodometry_in: {role: input, rung: odometry}; bringup remaps both to/{robot}/odometry. Neither learns the other's names — swap MACVO for OptiTrack and nothing downstream changes.doctorerror demanding an explicit override — the resolver never guesses. (droan.disparity_ininfull_default.yamlexists because two disparity sources are available.)Legibility: the resolved wiring is a first-class artifact.
airstack wiring(+ a per-run dump besideeffective_config.yaml) renders the dataflow graph — every connection with topic, type, QoS, and provenance (rung / auto-wired unique provider / override at stack.yaml:line), as terminal table + mermaid. Generated by the same resolver that builds the launch, so it cannot lie or rot. "What feeds DROAN?" = read stack.yaml + rung table + wiring report, not 80 lines of remap XML.doctor --livediffs the running graph against the resolved one (ros2 topic infoper connection: both endpoints present, QoS compatible) — catching the wired-but-silent class static checks miss.Cross-module checks:
provides/requiresnames + slot uniqueness gets ~90% of the value cheaply.doctorreports: two modules claiming one slot, unsatisfiedrequires, message-type mismatches vsairstack_msgs. No full ontology — names + types, checked at sync time.4. Modules that don't fit a slot (e.g., an end-to-end VLA planner)
A rigid slot taxonomy breaks on modules that swallow several slots — e.g., an end-to-end vision-language-action planner spanning perception → planning. Three refinements:
(a) Slots are contracts at narrow waists — rungs on a ladder. A conventional module taps in at one rung and out at the next; an end-to-end module taps in high and out low. Admission test: a rung exists only if modules on both sides plausibly vary independently — otherwise it's private wiring (§3 rule 2), deliberately not public API. The proposed set, derived from today's standard topics:
sensors/*Image/CameraInfo/PointCloud2/Imu; sensor-data QoS/{r}/sensors/front_stereo/…odometrynav_msgs/Odometry, map frame, min-rate class, reliable/{r}/odometryglobal_mapglobal_plannav_msgs/Path, map frame/{r}/global_plantrajectory(group)trajectory_segment_to_add,trajectory_override,set_trajectory_mode; up:tracking_point,look_ahead, completion %/{r}/trajectory_controller/*control_setpointinterface_status(group)is_armed,has_control,extended_state;robot_commandinbound/{r}/interface/*tasks/*provides_tasks, #380)/{r}/tasks/{navigate,takeoff,land,fixed_trajectory}safetydrone_safety_monitortopicsgossipPeerProfile+ attached payloadsNotes: rungs are topic groups with one contract (types + QoS + frames + rate class) — the trajectory rung is bidirectional, and per-topic rungs would recreate remap sprawl.
sensors/*is a convention, not a fixed list. Deliberate non-rungs: disparity/depth preprocessing stays private wiring (one consumer class; rung-ifying freezes an internal detail into public API); actuator commands below the interface are never a rung (safety floor). The list grows only via the §8 RFC process, with §11 drift reports as the discovery mechanism (three forks patching the same tap point = a missing rung).global_mapis the hardest: representations genuinely vary (voxel/ESDF/mesh), so it needs either one blessed representation + adapters or a narrower query interface — flagged, not hand-waved.(b) A module declares what it replaces:
The resolver vacates replaced slots (no double-publishing), verifies everything consumed is still provided, and warns about orphaned downstream consumers.
(c) Three tiers of decreasing verifiability, all first-class: slot filler (doctor verifies everything) → ladder-spanning
stack_module(doctor verifies wiring, not internals) → freeform (namespace + sensor bus; doctor checks collisions only).Non-negotiable safety floor across all tiers: command authority flows through the trajectory controller / interface layer, where arming, safety monitoring, and takeover live. A module emitting
trajectory_overrideinherits the whole safety apparatus free — a selling point, not a constraint. A module that must bypass it declaresproduces: [attitude_setpoint]and doctor flags it loudly as safety-critical.5. Version compatibility and test separation
Trunk's
docker-build.ymlalready pushes cosign-signed images tagged byVERSION— published images are the compatibility contract.Reusable workflow trunk owns (
.github/workflows/module-system-tests.yml). A module repo's entire CI:It: (1) checks out trunk at
airstack_ref; (2) checks out the module and runsairstack module add ./— the same overlay path a developer uses, so the module'sslot:wires it into the test bring-up's stack.yaml; (3) runs the existing suite unchanged viaairstack test -m "<marks>". The module repo never copies a test.System tests double as slot conformance tests.
tests/waypoint_checker.pyjudges the odometry track regardless of which planner produced it — passing it is the behavioral definition of a working global planner. Each slot contract names required marks (global_planner→waypoint_flight,autonomy;state_estimator→liveliness,sensors,takeoff_hover_land). The registry compat badge for (module version × AirStack version) is granted only when that set passes.GPU runners (system tests need GPU + sim license; trunk uses OpenStack ephemeral runners via
.github/orchestrator/):repository_dispatch {module_repo, module_ref, airstack_ref}; trunk runs on its own runners, posts a check-run back via a GitHub App. Secrets and licenses never leave trunk; bench time is gated/rate-limited.Cost ladder: every push →
unit+build_packagesin the published image (minutes, no GPU); PR/nightly →liveliness(+sensorswhere relevant),msairsimis the cheap bring-up; release/compat claim → full conformance set on GPU (stamps the badge); trunk-side nightly canary runs registered modules againstdevelopso breakage surfaces the day it lands.Metrics: module runs'
metrics.jsondiff against trunk's baseline via existingtests/parse_metrics.py— badges can carry e.g. "cross-track RMSE vs. default planner."6. Docker images: base + composed module layers
Never publish permutations. Trunk publishes one signed base per host type per version; composed images are built where used, from cached layers.
Three dependency tiers (declared in
module.yaml):package.xml+deps:; no Dockerfile in the module repo.Dockerfile.modulewritten againstARG BASE_IMAGE, never a fixed base.ghcr.io/…/<module>-overlay:<airstack_version>builtFROM ${BASE_IMAGE}; developers pull instead of build.Composition chain (
airstack module sync, deterministic order, grouped by target host):Module code changes rebuild nothing (source stays volume-mounted; dev loop =
bwsin the container, unchanged). A module's dep change rebuilds only its layer and above.modules.lockhashes dep declarations + records image digests →airstack upknows exactly when a rebuild is needed ("optitrack deps changed, rebuilding 2/4 layers") and compat badges record the digest chain they were earned with. CI uses the identical path with a registry-backed buildx cache (runners are ephemeral).Failure modes: (1) dep conflicts between modules →
doctorrunspip check/apt dry-run at compose time and names the fighting modules; (2) prebuilt overlays don't merge → a published overlay is used as-is only when it's the sole tier-3 module; otherwise build each from its fragment in chain order (overlay = cache, fragment = source of truth).Side effect: trunk Dockerfiles thin out as deps migrate into the modules that own them (MACVO networks, mocap SDKs).
7. Marketplace / registry
An
airstack-modules-indexrepo (rosdistro'sdistribution.yamlpattern): one YAML per module — name, repo URL, description, maintainer, CI-generated compat matrix.airstack module search|list|addreads it; MkDocs renders a catalog with badges. Getting listed = PR to the index = the quality gate (manifest valid, CI green, README template, license check).Release sets solve N×N module compatibility: periodically publish "AirStack 0.20 + these module versions, tested together" — one lockfile, blessed by a single CI run with all enabled. Individual badges mean "works with trunk"; release sets mean "works together." Most users pull a release set; à-la-carte is for developers. Release sets also drive versioned docs (§9).
8. Governance (most likely to decide success)
airstack_msgsruthlessly, as its own package. ROS 2 type hashes make msg mismatches fail silently (no connection), not gracefully.doctorcompares msg versions across the composed workspace.9. Docs
Hybrid, embedding automated. Pure link-outs to repo READMEs lose search, theming, and versioning (a GitHub README shows
main— wrong for a user on 0.19). Module docs ride the existingmikemachinery:docs/), declared viadocs:and validated against the README template at registration. The docs deploy workflow shallow-clones each registered module at a pinned ref intomodules/<name>/and splices nav (a fetch loop driven by the registry index — simpler thanmkdocs-multirepo-plugin; extends the existingsame-dirpattern).developembeds module default branches.repository_dispatch+ existing triggers + nightly. "Within a day, exact at releases."10. Smaller items (all decided)
assets: [{url, sha256, dest}], fetched to cache bymodule sync. No Git LFS in module repos.airstack module creategenerates the template repo (manifest, CI stub, launch, README); update.agents/skills/so AI-agent contributions build modules the new way instead of reintroducing the monolith.11. The researcher workflow: research in a fork, graduate to a module
Principle: research happens in a fork; modularity is a graduation step, not an entry fee. Demand manifests and CI during exploration and researchers will ignore the process — the design's job is making the fork cheap to extract from, which is decided on day one.
airstack init --release 0.20+module addwhat the experiments need. Stay pinned through the research.Dockerfile.robot), not moving files. One cheap discipline:airstack module create --in-tree my_plannerscaffoldsrobot/ros_ws/src/modules/my_planner/(normal colcon package, stub manifest, own launch) in the researcher's fork — zero CI/registry obligations, only the directory boundary from day one.airstack module doctor --driftdiffs the fork against the pinned release and classifies changes: contained in the module dir (fine) vs. trunk edits (extraction debt, listed).airstack module extract my_plannermoves the directory into a fresh template repo (manifest, CI pinned to their release, README skeleton) and emits the drift report as the task list. Then: conformance marks against the pinned version (same code — should pass immediately), then against current trunk (the deferred compat work, done once), then register..agents/skills/extract-moduleskill takes the manifest schema, contract specs, template, and drift report; the agent converts mechanically and verifies withdoctor+ the slot's conformance marks (it flies the drone until it passes). A second skill guides agents during Stage 1 to keep changes inside the module dir — cheap prevention that makes Stage 2 nearly free.12. Phased implementation plan
module.yamlschema + JSON Schema validation; reusablemodule-system-tests.yml; extract DFM2 disturbances as pilot (sim-side, no slot, lowest coupling).modules/overlay +modules.repos;module add|remove|sync|doctor; Docker composition chain +modules.lock; extract OptiTrack and the custom planner. Researcher tooling (§11):create --in-tree,doctor --drift,extract,extract-moduleskill.stack.yaml-driven bringup; collapselocal_*.launch.xmlinto presets. This phase is also where #380's vehicle/fleet/top-level configuration layer lands — its config surface consumes the slots defined here.Heterogeneous deployment configuration and cross-embodiment platforms are specified in RFC #380; its cross-embodiment part is sequenced after phases 1–4 are proven.
Ranked success factors: (1) the CI-against-published-images loop, (2) contract governance/deprecation policy, (3) maintainer lifecycle policy. Everything else is buildable later; these determine whether the ecosystem stays trustworthy.
Appendix: trunk facts an implementer needs
.envVERSION="0.19.0-alpha.9"tags docker images;check-version-increment.ymlgates PRs on semver bump..github/workflows/docker-build.ymlbuilds, pushes, cosign-signs all compose images on merge tomain/developwhenVERSIONchanges.tests/system/(marks intests/pytest.ini:unit,build_docker,build_packages,integration,liveliness,sensors,takeoff_hover_land,autonomy,waypoint_flight); harness intests/harness/+tests/conftest.py; unit-test packages listed intests/colcon_unit_test_packages.yaml; metrics viatests/parse_metrics.py(diff-vs-baseline, exits 1 on regression); waypoint pass/fail via standalonetests/waypoint_checker.py.runs-on: [self-hosted, airstack-ephemeral]→ OpenStack ephemeral VMs via.github/orchestrator/(spawn loop, JIT tokens, cloud-init, reap loop). Fork PRs blocked from runners.robot/ros_ws/src/autonomy_bringup/launch/robot.launch.xml, role-dispatched byAUTONOMY_ROLE(full|onboard|offboard); per-layer bringups likerobot/ros_ws/src/local/local_bringup/launch/(the variant explosion).deploy.replicas: ${NUM_ROBOTS};ROBOT_NAME/ROS_DOMAIN_IDresolved at container start byrobot/docker/.bashrc+robot/docker/robot_name_map/resolve_robot_name.py./{robot}/odometry,/{robot}/global_plan,/{robot}/trajectory_controller/{trajectory_override,trajectory_segment_to_add,look_ahead,tracking_point,set_trajectory_mode},/{robot}/tasks/{navigate,takeoff,land,fixed_trajectory},/{robot}/interface/*— seedocs/robot/autonomy/integration_checklist.md.mikeversioning,same-dirplugin for out-of-tree READMEs, deploy workflows per branch.simulation/), OptiTrack integration (natnet_ros2exists in trunk), custom global planner.From a design brainstorm session (2026-08-04). Companion: RFC #380 — Heterogeneous AirStack. Comments and pushback welcome — especially from prospective module authors and registry maintainers.
All reactions