Project Emerge

Description

Project Emerge is a modular, open-source toolchain for practical multi-robot demonstrations based on aggregate computing principles. It is designed for research, demonstrations, and educational robotics.

Swarm coordination uses ScaFi, an aggregate programming framework for expressing collective behavior through computations over each robot's local state and its neighbors. In this stack, a Scala runtime executes the aggregate programs for the fleet and publishes motor commands over MQTT.

A real-time React dashboard visualizes and controls the robots, a Python neighborhood service computes their communication neighborhoods, and VisionSystem localizes ArUco markers using calibrated USB cameras. MQTT connects localization, coordination, monitoring, and robot actuation. The dashboard also provides fleet configuration and firmware OTA updates.

When hardware is unavailable, the bundled robot emulator supplies simulated poses and responds to motor commands, allowing the same dashboard and aggregate runtime to be used without physical robots or cameras.

Project structure

flowchart LR
  AR["Aggregate Runtime<br/>Scala / ScaFi"]
  NS["Neighborhood System<br/>Python / Flask"]
  D["Dashboard & Gateway<br/>React / Node.js"]
  MB[("MQTT Broker<br/>Mosquitto")]
  V["VisionSystem<br/>Python / OpenCV"]
  S["Robot Emulator<br/>Python"]
  R["Physical Robots"]

  V -- "poses" --> MB
  MB -- "poses" --> NS
  NS -- "neighbors" --> MB
  MB -- "poses, neighbors, formation" --> AR
  AR -- "motor commands" --> MB
  MB -- "state & telemetry" --> D
  D -- "configuration & manual commands" --> MB
  MB -- "commands" --> R
  R -- "telemetry" --> MB
  MB -- "commands" --> S
  S -- "simulated poses & telemetry" --> MB
  R -. "camera observations" .-> V

Loading

VisionSystem and the emulator provide alternative pose sources. VisionSystem can run on one machine or split camera acquisition and pose fusion across multiple machines; see the VisionSystem guide.

Polyglot monorepo for the Project Emerge services. Deployable applications live in apps/ regardless of implementation language; infrastructure shared by the stack lives in infra/.

Repository layout

apps/
  dashboard/          React dashboard and Node MQTT/WebSocket gateway
  aggregate-runtime/  Scala/ScaFi aggregate-computing runtime, publishes robot actuation over MQTT
  neighborhood-system/ Python/Flask neighborhood computation and HTTP configuration
  vision/             Python/OpenCV localization, calibration, and distributed camera nodes
  robot-emulator/     Python robot simulator used instead of physical vision input
packages/             Optional language-specific shared libraries
infra/mosquitto/      MQTT broker configuration
compose.yaml          Local/edge service orchestration
Makefile              Language-neutral entry points
tools/                Supporting utilities, including battery-life analysis
images/               Project branding

Components

Component	Role
Dashboard	Live swarm visualization, manual driving, formation selection, marker mapping, and OTA uploads; Node.js bridges the browser to MQTT.
Aggregate runtime	Executes ScaFi formation programs and converts their outputs into differential-wheel commands.
VisionSystem	Camera calibration, ArUco detection, multi-camera pose fusion, and publication of mapped robot poses.
Robot emulator	Simulates robot motion and MQTT feedback for demonstrations without hardware.
Neighborhood system	Consumes robot poses and publishes full or radius-based neighbor lists; exposes HTTP configuration on port 5000.
Mosquitto	Shared MQTT messaging for all services and robots.

Monorepo conventions

The root npm workspace lists JavaScript applications explicitly. Do not replace it with an apps/* glob: future Python and Scala applications are not npm packages.

Each deployable owns its language manifest, tests, and Dockerfile. A Python service should own its pyproject.toml and lockfile; a Scala service should own build.sbt, project/, and the standard src/main and src/test trees. Add its native build and test commands to the aggregate Makefile targets when the service is introduced.

Local dashboard development

Requirements: Node.js 24 and npm 11.

cp .env.example .env
make bootstrap
npm run dev

Open http://localhost:5173. Vite forwards /api and /ws to the gateway on port 8787. The root .env remains the configuration source after the dashboard's move into apps/dashboard.

The equivalent direct commands remain available:

npm test
npm run build
npm start

Container stack

Start the complete hardware stack, including the USB-camera vision runtime, with:

make up
make ps

The equivalent direct command is docker compose up --build -d. Vision reads apps/vision/config.local.json, loads and updates calibration artifacts under apps/vision/calibrations/, connects to mosquitto:1883, and persists its runtime state and diagnostics in named volumes. On Linux, the container receives access to V4L2 USB camera devices (major number 81) and supports both numeric camera sources and stable /dev/v4l/by-id/... paths.

Before the first hardware start, create the local camera configuration and calibrations as described in apps/vision/README.md. The runtime remains alive and retries if a configured USB camera is temporarily disconnected.

To run the robot simulator without loading the physical vision system, use:

make up-simulator

This target first stops an already-running vision container and then starts only the common services plus simulator. Do not use a bare docker compose --profile simulator up: Compose profiles are additive, so that form would also start every ordinary service, including Vision. The direct equivalent of the safe target is:

docker compose stop vision
docker compose --profile simulator up --build -d \
  dashboard aggregate-runtime simulator neighborhood-system mosquitto

The dashboard is available at http://localhost:8787; MQTT clients connect to mqtt://localhost:1883. Override the published host ports with DASHBOARD_PORT and MQTT_PORT in the root .env when necessary. Containers communicate over the emerge-backplane network, where the broker has the stable address mosquitto:1883.

The broker intentionally allows anonymous clients because the current robots operate on a trusted LAN. Port 1883 must not be exposed to an untrusted network without adding authentication, ACLs, and preferably TLS.

Stop the stack without deleting state:

make down

dashboard-firmware stores uploaded OTA images and manifests, while mosquitto-data stores retained MQTT messages and persistent sessions. vision-state contains the last accepted runtime configuration and vision-diagnostics contains the structured VisionSystem logs. All named volumes survive container recreation and normal docker compose down. Running docker compose down -v permanently deletes these volumes and their data.

Configuration

Variable	Default	Description
MQTT_URL	mqtt://192.168.8.1:1883	Broker used by standalone dashboard development. Compose always uses mqtt://mosquitto:1883.
MQTT_USERNAME / MQTT_PASSWORD	—	Optional credentials for an external broker.
HTTP_PORT	8787	Standalone gateway port.
FIRMWARE_DIRECTORY	firmware	Standalone firmware storage directory. Compose mounts its named volume separately.
DASHBOARD_PORT	8787	Dashboard port published by Compose.
MQTT_PORT	1883	MQTT port published by Compose.
VITE_OTA_SERVER	Current dashboard host and port	Optional OTA address suggested by the UI.
VITE_GATEWAY_URL	Same-origin /ws	Optional WebSocket gateway override.

MQTT behavior

The gateway subscribes to /pose/+, /telemetry/+, /imu/+, /config/ota, /config/motors, /config/aruco-map, and /config/formation. Motor configuration is fleet-wide: it is published once with QoS 1 and retention on /config/motors, so every robot — including robots that join later — applies the same settings.

Manual dashboard driving publishes normalized differential-wheel commands to /motors/{id} while the joystick is held. Moving commands use the payload {"Move":{"left":-1..1,"right":-1..1}} at 10 Hz with QoS 0; release and safety stops use "Stop" with QoS 1. Motor commands are never retained. Robot telemetry separately reports motor state as Motoring or Stopped. Robot firmware should independently stop both wheels when fresh commands have not arrived for 300 ms so a broken browser or network connection cannot leave the robot moving.

The aggregate-runtime service publishes the same /motors/{id} topic and {"Move":{...}}/"Stop" payload shape when it drives a robot swarm through an aggregate-computing program, using the robot's device ID formatted as 6 uppercase hex digits.

Settings → Marker mapping lets an operator register which ArUco marker ID (0-49) corresponds to which robot device ID. The mapping is retained as a single JSON object ({"<markerId>": "<deviceId>", ...}) on /config/aruco-map with QoS 1, so it survives broker restarts. VisionSystem consumes this mapping and publishes each mapped marker on /pose/{deviceId} while retaining its detailed raw-marker pose under the vision/<site>/<system_id>/pose/{markerId} namespace.

The dashboard's home page has a Formation & parameters panel for choosing a swarm formation (pointToLeader, vShape, lineShape, circleShape, squareShape, verticalLineShape, stop), a leader robot, and that formation's numeric parameters. Applying it retains a single JSON object ({"program": "<name>", "leaderId": "<deviceId>" | null, "params": {"<key>": <number>, ...}}) on /config/formation with QoS 1. aggregate-runtime is the only subscriber: it merges program, the resolved leader (decoded from the hex device ID), and params into the sensor map its ScaFi formation programs read via sense(...).

OTA updates

For OTA updates, provide a dashboard host:port reachable by the robots, the firmware version, and the compiled .bin file in Settings → OTA update. Use the dashboard's LAN address rather than localhost. The gateway stores the image, retains the OTA server on /config/ota, and publishes a non-retained /ota/check/{id} command to each discovered robot.

Credits

Contributors:

Contributions are welcome: open an issue, submit a pull request, or suggest an improvement.