Fast C++ implementations of Farama mpe2 (Multi-Agent Particle Environments) with a PettingZoo-compatible Python API. Drop-in replacement for mpe2 with 7–55× faster rollouts.
Every environment in mpe2 has a C++ implementation built on a shared core::BaseEnv + core::World + core::Scenario abstraction, exposed to Python via pybind11 and wrapped to match PettingZoo's ParallelEnv and AEC APIs.
| Environment | Description | Step speedup vs mpe2 |
|---|---|---|
| Simple | 1 agent, 1 landmark — the basic single-agent task | 13.56× |
| SimpleSpread | 3 cooperative agents covering 3 landmarks | 22.45× |
| SimpleReference | 2 agents with 10-word communication to identify goals | 15.53× |
| SimpleSpeakerListener | Asymmetric speaker/listener task | 15.68× |
| SimpleAdversary | 1 adversary + N good agents racing to a goal | 15.16× |
| SimpleTag | 3 adversaries chase 1 good agent (predator-prey) | 18.96× |
| SimplePush | 1 adversary tries to push 1 good agent off-goal | 13.43× |
| SimpleFormation | 4 agents form a moving formation (Hungarian matching) | 17.56× |
| SimpleLine | 4 agents arrange themselves along a line | 16.30× |
| CollectTreasure | 6 collectors + 2 deposits + 6 respawning treasures | 38.03× |
| SimpleWorldComm | Leader adversary + comms + forest visibility | 54.81× |
| SimpleCrypto | Alice/Bob/Eve — encrypted message passing | 12.47× |
Average step speedup: 19.2× | Range: 7.2× – 54.8×
Full results (reset/step/episode breakdown, hardware, methodology) in BENCHMARKS.md.
Requires Python 3.10+, CMake 3.15+, a C++17 compiler, and uv for Python dependency management.
git clone https://github.com/rob-pitkin/cpp-mpe2.git
cd cpp-mpe2
uv sync
uv run cmake -S . -B build
uv run cmake --build build -j8This produces _simple_core.so, _simple_spread.so, etc. inside build/, which the Python wrappers locate at import time.
The Python API is intentionally a drop-in replacement for mpe2. Swap your import and everything else stays the same.
# Before (mpe2)
from mpe2 import simple_spread_v3
env = simple_spread_v3.parallel_env()
# After (cpp-mpe2)
from cpp_mpe2.simple_spread.simple_spread import parallel_env
env = parallel_env()Every environment lives at cpp_mpe2.<env_name>.<env_name> and exposes three constructors:
| Constructor | Returns | Use for |
|---|---|---|
parallel_env(**kwargs) |
PettingZoo ParallelEnv |
All agents step in parallel — the fast path for RL training |
env(**kwargs) |
PettingZoo AEC env | Turn-based stepping, one agent at a time |
raw_env(**kwargs) |
Same as parallel_env |
Alias matching mpe2's naming convention |
from cpp_mpe2.simple_spread.simple_spread import parallel_env, env, raw_env
from cpp_mpe2.simple_world_comm.simple_world_comm import parallel_env as swc
from cpp_mpe2.simple_tag.simple_tag import parallel_env as tag
# ... one import per env, same patternAll agents act simultaneously each step. This is the format most MARL training loops want.
from cpp_mpe2.simple_spread.simple_spread import parallel_env
env = parallel_env(max_cycles=25)
obs, infos = env.reset(seed=42)
while env.agents:
actions = {a: env.action_space(a).sample() for a in env.agents}
obs, rewards, terminations, truncations, infos = env.step(actions)
# train your policy hereenv.agents becomes empty when the episode ends (truncation or termination), which is the natural loop exit condition.
One agent acts per step, in a fixed cycle order. Useful when your algorithm needs strict sequential semantics.
from cpp_mpe2.simple_spread.simple_spread import env as aec_env
env = aec_env(max_cycles=25)
env.reset(seed=42)
for agent in env.agent_iter():
obs, reward, termination, truncation, info = env.last()
action = None if (termination or truncation) else env.action_space(agent).sample()
env.step(action)Spaces are per-agent (not shared) because several environments are asymmetric:
env = parallel_env()
env.reset(seed=0)
for agent in env.possible_agents:
print(agent, env.observation_space(agent), env.action_space(agent))
# agent_0 Box(-inf, inf, (18,), float32) Discrete(5)
# agent_1 Box(-inf, inf, (18,), float32) Discrete(5)
# agent_2 Box(-inf, inf, (18,), float32) Discrete(5)For asymmetric envs (SimpleSpeakerListener, SimpleWorldComm, SimpleCrypto, SimpleAdversary, SimplePush), different agents have different shapes:
from cpp_mpe2.simple_world_comm.simple_world_comm import parallel_env
env = parallel_env(); env.reset(seed=0)
env.action_space("leadadversary_0") # Discrete(20) — movement × comm
env.action_space("adversary_0") # Discrete(5) — movement only
env.observation_space("agent_0") # Box (28,) — good agent
env.observation_space("adversary_0") # Box (34,) — includes leader's commEvery env supports both discrete and continuous action spaces via the continuous_actions kwarg (matches mpe2):
env = parallel_env(continuous_actions=True)
env.reset(seed=0)
env.action_space("agent_0") # Box(0.0, 1.0, (5,), float32)
# Movement actions are interpreted as [no_op, left, right, down, up] forces.Pass render_mode="rgb_array" for an (H, W, 3) numpy frame or "human" for an interactive pygame window. Default is no rendering.
env = parallel_env(render_mode="rgb_array")
env.reset(seed=0)
frame = env.render() # numpy array, shape (700, 700, 3), dtype uint8Rendering is lazy — if render_mode=None (the default), no pygame state is created and there's zero per-step overhead. Always omit render_mode for training runs.
All envs accept these (with sensible defaults):
| Kwarg | Type | Notes |
|---|---|---|
max_cycles |
int |
Episode truncation length (default 25) |
continuous_actions |
bool |
Box action space if True, Discrete otherwise |
render_mode |
str | None |
"human", "rgb_array", or None |
dynamic_rescaling |
bool |
Scale entity radii with the camera view |
Plus per-env counts: SimpleSpread takes N agents/landmarks, SimpleWorldComm takes num_good / num_adversaries / num_obstacles / num_food / num_forests, etc. Check the __init__ signature of any env's parallel_env.
For centralized critics, env.state() returns a flat concatenation of all per-agent observations:
env = parallel_env()
env.reset(seed=0)
state = env.state() # np.float32 array, shape (sum of all agent obs sizes,)Because env(...) returns a parallel_to_aec_wrapper-wrapped env, it composes with PettingZoo's standard wrappers:
from pettingzoo.utils import wrappers
from cpp_mpe2.simple_spread.simple_spread import env
e = env()
e = wrappers.OrderEnforcingWrapper(e)
e = wrappers.AssertOutOfBoundsWrapper(e)cpp-mpe2/
├── core/ # Shared C++ runtime
│ ├── entity.h # Agent, Landmark base classes
│ ├── world.{h,cpp} # Physics integration, collisions
│ ├── scenario.h # Scenario interface (env-specific logic)
│ ├── base_env.{h,cpp} # Step/reset loop, action decomposition
│ └── types.h # ObservationMap, ActionMap, State, etc.
├── cpp_mpe2/ # Python package
│ ├── core/ # Python-side Agent/Landmark/World shims for rendering
│ ├── simple/ # One subdir per environment
│ │ ├── simple.py # PettingZoo wrapper (ParallelEnv + AEC + render)
│ │ └── cpp/ # C++ scenario + env + pybind11 bindings
│ │ ├── simple_scenario.{h,cpp}
│ │ ├── simple_env.{h,cpp}
│ │ └── bindings.cpp
│ ├── simple_spread/ ... # (same pattern for all 12 envs)
│ └── benchmark_*.py # One benchmark script per env
├── tests/ # 411 tests covering API, AEC, equivalence, render
├── CMakeLists.txt # Builds one pybind11 module per env
├── pyproject.toml
└── BENCHMARKS.md # Detailed perf analysis
A few design choices worth knowing if you're extending this:
- One
core::BaseEnvfor all envs. It handles the step/reset loop, max-cycles truncation, observation/reward collection, and discrete-action decomposition (action % 5→ movement,action / 5→ comm one-hot). Adding a new env means writing aScenariosubclass; the loop is shared. core::Scenariointerface hasmake_world,reset_world,reward,observation, plus optional hooks:global_reward(for local/global blending),is_terminal, andpost_step(for envs like CollectTreasure that mutate world state between physics and reward computation).- Landmark color is a render-only concept. It's set Python-side at
__init__, never round-tripped through C++. Forgetting this causesNone * 200render crashes — every env must explicitly assign landmark colors in its Python wrapper. - No new C++ for asymmetric action spaces. The
Discrete(20)leader action insimple_world_command theDiscrete(4)pure-comm action insimple_cryptoboth worked out of the box becauseBaseEnv::stepalready handled the generalmovement % mdim/action / mdimdecomposition. Action space asymmetry lives entirely in the Python wrapper's_action_spacesdict. Landmark::subtypeis a generic int tag used bysimple_world_comm(obstacle/food/forest) and could be reused by other envs that pack multiple landmark categories into one vector.
uv run pytest tests/ # All 411 tests
uv run pytest tests/test_simple_spread/ # One env's tests
uv run pytest -k equivalence # Just the mpe2-parity checksEach environment has three test files:
test_api.py/test_equivalence.py—parallel_api_test+ shape/action-space parity withmpe2test_aec.py— AEC API compliance viapettingzoo.test.api_testtest_render.py— RGB array + human-mode rendering
uv run python cpp_mpe2/benchmark_simple_spread.pyEach benchmark runs 1M resets, 1M steps, and 100K episodes against both backends and reports the per-op speedup. See BENCHMARKS.md for the full table and per-environment analysis.
- Build: CMake 3.15+, C++17 compiler, pybind11 (auto-installed via uv),
munkres-cpp(vendored undercore/third_party/for Hungarian matching inSimpleFormation/SimpleLine) - Runtime: Python 3.10+,
pettingzoo>=1.24,gymnasium>=0.29,numpy>=1.24,pygame-ce(for rendering) - Dev:
pytest,mpe2==1.1.0(used as the equivalence-test oracle)
All managed via uv sync from pyproject.toml.