Linux Native Wrappers

Linux shared libraries that let Space Engineers 1 and Space Engineers 2 load their Windows native DLLs. Most wrappers are C++17 shims around a custom PE (Portable Executable) loader. They load the original DLL, translate calls between the System V and Microsoft x64 ABIs, and implement the Win32 and CRT functions those DLLs need. Events and semaphores use ntsync when /dev/ntsync is available, with Linux synchronization primitives as the fallback.

PlatformWindows is an exception. It is a native presenter adapter and does not load a Windows DLL.

Libraries built

Output	Game	Purpose
libHavok.so	SE1	Loads Havok.dll for physics
libRecastDetour.so	SE1	Loads RecastDetour.dll for navmesh
libVRageNative.so	SE1	Loads VRage.Native.dll for voxels
libD3DCompiler.so	SE1	Loads d3dcompiler_47.dll for shader compilation
libVRage.KytheraV2.Native.so	SE2	Loads VRage.KytheraV2.Native.dll and its Kythera_*.dll imports for AI navigation
libVRage.Platform.Windows.Native.so	SE2	Provides the native presenter adapter
libVRage.Slug.Native.so	SE2	Loads VRage.Slug.Native.dll for text rendering
libVRage.Physics.Native.so	SE2	Loads VRage.Physics.Native.dll for physics
libVRage.Voxels.Native.so	SE2	Loads VRage.Voxels.Native.dll for voxels

CMake also builds the generate_pe_sidecar and d3dcompiler_tool command-line tools. Release archives contain only the shared libraries in the table.

Runtime setup and limits

Loader-backed wrappers expose Init(dllPath, sidecarPath). The first argument is the path to the matching Windows DLL. The second is an optional cache path for its ELF sidecar; the parent directory must be writable when the sidecar needs to be created or replaced. Call Init before using the wrapper's other exports. Physics, Slug, Voxels, and KytheraV2 enforce this explicitly. Wrappers do not search for game installations or DLLs; the only other DLLs they load are PE imports found next to the DLL passed to Init, described below.

This loader implements the Windows APIs used by these DLLs. It is not a general PE or Win32 runtime. It supports x86-64 PE32+ images and selected imports. A PE import that is not a shimmed module and sits next to the importing DLL is loaded from there, linked and attached first, and gets its own sidecar next to the importer's one. KytheraV2 loads 18 Kythera_*.dll files this way. Unsupported imports use trap stubs, MSVC exception boundaries abort the process, and loaded PE images cannot be unloaded safely. These failures are intentional: continuing would leave the process in an unknown state.

Generated wrappers

The Havok, Physics, Voxels, and KytheraV2 wrappers are generated from decompiled C# DllImport declarations. Their generated C++ files are committed, so building the project does not require decompiled sources.

Each generator takes one decompiled source root. The root may contain the assembly directly, under src/, or be the assembly directory itself.

python3 tools/generate_havok_wrapper.py /path/to/se1/decompiled
python3 tools/generate_physics_wrapper.py /path/to/se2/decompiled
python3 tools/generate_voxels_wrapper.py /path/to/se2/decompiled
python3 tools/generate_kythera_wrapper.py /path/to/se2/decompiled

These commands rewrite src/Havok.cpp, src/Physics.cpp, src/Physics.exports, src/Voxels.cpp, and src/KytheraV2.cpp as applicable. Run the generator smoke tests after regeneration:

ctest --test-dir build --output-on-failure

Havok callbacks

Havok accepts Microsoft-ABI callback pointers, while managed delegates use the System V ABI on Linux. The generated wrapper uses one fixed bridge for each static semantic callback, serialized bridges for the two callbacks that live only for a native call, and three shared bridges for phantom shapes. Phantom callbacks are routed through a map keyed by the native shape pointer.

KytheraV2 callbacks

KytV2_CreateCore receives the managed callback table. The wrapper passes the bridge its own long-lived copy, with an ms_abi bridge to the managed RequestBackgroundWork function and its argument unchanged. The two profiler callbacks are Optick exports, which have no Linux build, so the copy gets no-op ms_abi stubs in their place. Kythera returns navmesh triangles, paths and cover points in CoTaskMemAlloc buffers. The shim allocates them with malloc, because .NET frees them with free through Marshal.FreeCoTaskMem.

Decorated SE2 exports

Some MSVC-decorated Physics and Slug exports contain @, which GNU ELF tools interpret as symbol-version syntax. The source uses equal-length $ placeholders. Post-build scripts replace those names in the ELF dynamic string table and rebuild its System V symbol hash.

Physics records exact placeholder-to-export mappings in src/Physics.exports. Slug has 14 known exports and derives each final name by replacing $ with @, so it does not need a manifest. CMake runs both renamers automatically and fails the build if the expected symbols are not found.

Both targets also compile with -fno-reorder-blocks-and-partition. At -O2 and above GCC otherwise splits cold paths into a separate section and appends .cold to the mangled symbol name, producing identifiers the GNU assembler cannot parse.

Building locally

The direct CMake build requires x86-64 Linux, CMake 3.13 or newer, Python 3, and a C++17-capable g++. The supplied Makefile uses a CMake preset and therefore requires CMake 3.21 or newer. The packaging script also uses Bash, GNU Make, GNU tar, GNU coreutils, and GNU binutils (readelf, objcopy, strip).

make
ls build/*.so
make clean

make configures build/ with the default preset, builds every target, and does not select a CMake build type. Use a direct CMake configuration when you need a specific build type:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j "$(nproc)"
ctest --test-dir build --output-on-failure

CTest is enabled by default. Pass -DBUILD_TESTING=OFF when only the libraries and command-line tools are needed.

TLS dialect

Every target is compiled with -mtls-dialect=gnu2. The Win32 and CRT shims are ms_abi functions, and GCC 12 and 13 assume the traditional call __tls_get_addr sequence preserves RDI and RSI inside them, which it does not. In an optimized shared library that turned a thread_local access in a shim into a call with a corrupted argument; the CRT qsort bridge sorted the GOT instead of the caller's array, and CreateEventW never ran the ntsync initializer. TLS descriptors resolve thread-local addresses with a call that preserves every register except RAX, so the shims are safe regardless of what the compiler keeps in RDI or RSI. Debug builds at -O0 were never affected. The tls_dialect test fails if a library imports __tls_get_addr again.

Tests

A normal build has these self-contained tests:

Test	Coverage
se1_generators	SE1 generator callback patterns and committed source markers
se2_generators	SE2 generators with synthetic C# declarations
package_symbols	Sample ELF: Release symbol split, shipped hashes/build IDs, patched exports, behavior and reproducibility
thread_id_dll_attach	PE thread attach, detach, TLS, and FLS lifetime
windows_memory	Windows memory and runtime behavior
crt_qsort	Shared-library ABI bridge; repeated, nested, concurrent and empty/single-item sorts
tls_dialect	No wrapper library imports __tls_get_addr (see above)
ntsync_init_event, ntsync_init_semaphore	Each API initializes ntsync independently in a fresh process and completes a shared-library round trip when the device is accessible
ms_abi_tls	An ms_abi function in a shared library keeps register values across a thread-local access, with dynamic TLS resolution forced

No C++ test compiles the wrapper sources into its executable; they link or load shared libraries instead. In an executable the compiler uses static TLS models, so a miscompiled dynamic TLS access in the libraries would never show up. thread_id_dll_attach needs the test-only hooks behind NATIVE_WRAPPERS_TESTING; it links build/tests/libNativeWrappersTesting.so, the shared sources built as a library with those hooks. The release packaging and the build/*.so deploy globs skip it.

With BIN64 pointing at the game's Bin64, the Havok tests also run against the real Havok.dll:

Test	Coverage
havok_unwind	Crash backtraces name PE frames and the wrapper export
havok_memory	Repeated shape creation does not leak Havok memory
havok_endofstep_detects	The havok_endofstep_test oracle finds the dangling contact manager that crashes SE1 inside FinishMtStep (SE1-0015); expected to report a violation
havok_endofstep	The same stress run stays clean with --fix, the Bugfixes plugin's workaround ported into the harness

havok_endofstep_test is also a standalone stress harness: it steps several worlds the way MyPhysics.StepWorldsParallel does, churns bodies with the game's lifetime sequences and checks the end-of-step callback list before every FinishMtStep. See the usage comment in tests/havok_endofstep.cpp for the scenarios and tests/havok_endofstep_loop.sh for a seeded soak. The game gets the fix from the Bugfixes plugin (HavokEndOfStepCallbackPatch), not from the wrappers; keep the harness copy of it in sync. If a future Havok.dll fixes the bug, havok_endofstep_detects starts failing, which is the signal to retire the plugin patch.

Run them with:

ctest --test-dir build --output-on-failure

CTest runs ms_abi_tls with GLIBC_TUNABLES=glibc.rtld.optional_static_tls=0. Without this setting, glibc can allocate the small probe in static TLS, so fresh threads alone do not exercise the dynamic resolver. Use the same environment setting when running ms_abi_tls_test directly.

The generator tests are smoke tests. They do not compare a full regeneration from the game assemblies with every committed wrapper.

Run the packaging check independently with python3 tests/package_symbols.py. It compiles a tiny sample library and needs GNU binutils; it builds no wrappers.

Packaging locally

build_and_package.sh builds the project, runs CTest, and packages the wrappers in both configurations. It produces the six archives used by the release workflow:

./build_and_package.sh
ls dist/*.tar.gz

Set BUILD_JOBS=4 to bound compiler concurrency on memory-constrained machines. Packaging verifies all six archives and writes dist/release-manifest.json with their exact SHA-256 hashes and source commit. Repeat the check independently with python3 tools/verify_release.py dist; it reads ELF metadata without loading wrappers.

It builds and packages Release first, then Debug, starting each configuration from make clean. The archives land in dist/, which is excluded from Git, and build/ is left holding the Debug libraries, so local development gets unoptimized binaries with full symbols and usable backtraces without another build. The build workflow runs the same script, so local archives have the same names and layout as published archives. Binary contents still depend on the local compiler and system libraries.

PE sidecars

Loader-backed wrappers can write an ELF sidecar to the cache path supplied by their caller. The sidecar keeps exported names, synthetic names for runtime functions, and supported unwind metadata available to Linux crash tools after the process exits. It does not contain PDB or private symbols.

Create a persistent sidecar directly when a saved core or another tool needs to reopen it later:

build/generate_pe_sidecar /path/to/Havok.dll /path/to/cache/Havok.dll

The loader checks the sidecar version, source size, and full-file FNV-1a fingerprint before using its mapped image. It atomically replaces a stale sidecar. If the cache path cannot provide a usable sidecar, the loader falls back to the raw PE image. Sidecar generation supports x86-64 PE32+ images and a subset of Windows x64 unwind records.

DLL-backed tests

The optional tests below use proprietary game DLLs. CI does not provide those DLLs, so it skips these tests. Set one or both directory options while configuring the build:

cmake -S . -B build \
  -DBIN64=/path/to/SpaceEngineers/Bin64 \
  -DGAME2=/path/to/SpaceEngineers2/Game2
cmake --build build
ctest --test-dir build --output-on-failure

Option and required file	Added tests
BIN64/d3dcompiler_47.dll	d3dcompiler_preprocess, d3dcompiler_diagnostics, d3dcompiler_forward_include
BIN64/RecastDetour.dll	recast_detour
BIN64/Havok.dll	pe_sidecar_generation, havok_unwind, havok_memory
GAME2/VRage.Physics.Native.dll	physics_init
GAME2/VRage.Voxels.Native.dll and VRage.Slug.Native.dll	voxels_teb
GAME2/VRage.KytheraV2.Native.dll	kythera_lifecycle

voxels_teb calls the Voxels planet shape, then Slug, then the planet shape again on one thread. The second Voxels call must set up its own TEB again, because the planet shape reads a thread-local variable of the DLL.

kythera_lifecycle runs the startup, update and teardown calls of KytheraCore, builds a ground navmesh on a test quad and queries a path on it, then updates a surface agent while GS points at another TEB, as it does after a Physics or Slug call on the same thread. Navmesh tiles build on background threads that call KytV2_DoBackgroundWork when the bridge asks for work. A last thread exits with GS on a foreign TEB, which the bridge's thread detach callbacks must survive. It fails on any unknown or ordinal import.

The Havok crash harness can also be run directly:

build/havok_crash_test /path/to/SpaceEngineers/Bin64/Havok.dll --trace
gdb --args build/havok_crash_test /path/to/SpaceEngineers/Bin64/Havok.dll

The --trace form checks the unwind trace and exits normally. Without --trace, the harness leaves the crash unhandled for GDB or core testing.

Releases

The build workflow runs for pushes to main. It also runs when a non-draft pull request is opened, updated, reopened, or marked ready for review. The workflow invokes build_and_package.sh, which builds and tests the Release and Debug configurations and packages the shared libraries at the root of one archive per game and configuration. Each archive holds only the wrappers its game version loads, as listed in Libraries built:

Asset	Game	Configuration
se1-native-wrappers.tar.gz	SE1	Release (-O3 -DNDEBUG)
se2-native-wrappers.tar.gz	SE2	Release (-O3 -DNDEBUG)
se1-native-wrappers.debug.tar.gz	SE1	Debug (-O0 -g)
se2-native-wrappers.debug.tar.gz	SE2	Debug (-O0 -g)
se1-native-wrappers.symbols.tar.gz	SE1	Split symbols matching the optimized Release libraries
se2-native-wrappers.symbols.tar.gz	SE2	Split symbols matching the optimized Release libraries

A push to main publishes a public v1.0.<run> release and marks it as the latest release. For a non-draft pull request, the workflow maintains a draft release with the tag pr-<number>. Draft publication requires repository write permission, which fork pull requests do not receive from the standard token.

Pulsar for Linux and Magnetar consume se1-native-wrappers.tar.gz by asset name. The Debug archives retain symbols for debugging. These per-game archives replaced the single combined linux-native-wrappers.tar.gz, which releases no longer carry.

Matching Release symbols

Release compilation retains -O3 -DNDEBUG and adds -g plus an ELF build ID. Packaging copies the libraries to staging, extracts DWARF into .so.debug files, then runs strip --strip-debug and adds a GNU debuglink to each staged library. Runtime downloads contain no DWARF. These operations change ELF file bytes while preserving executable code and runtime behavior; they do not strip the original compiler outputs in build/. The packaging script subsequently cleans that directory before its existing Debug build.

Each symbols archive contains the matching .so.debug files and symbols.json. The manifest records the source commit, ELF build IDs, symbol-file SHA-256 hashes, and SHA-256 hashes of the shipped libraries after stripping and debuglink. Compare the runtime library using sha256sum libHavok.so and readelf -n libHavok.so before selecting symbols. Extract its exact release's symbols archive alongside the runtime libraries (or put the .debug files in their .debug/ subdirectory) so GDB can follow the GNU debuglinks.

Retain the exact runtime and symbols assets together for every release under investigation. A separately compiled Debug (-O0) archive cannot symbolize the optimized Release code, and rebuilding the same source later does not guarantee matching binaries. These assets enable symbolication for new releases; they do not recover DWARF omitted from older releases. Loaded Windows DLLs still require their own matching symbols; wrapper DWARF does not supply those.

License

The project is MIT licensed; see LICENSE. The bundled Linux ntsync UAPI header in src/linux/ntsync.h carries its own GPL-2.0 WITH Linux-syscall-note SPDX identifier.