Skip to content

README.md

Latest commit

 

History

History
155 lines (101 loc) · 9.14 KB

README.md

File metadata and controls

155 lines (101 loc) · 9.14 KB
Raw

Engine Docker Build v2

This directory contains the Docker engine build scripts.

The container images bundle all required toolchains, dependencies, and configuration. This allows reliably compiling the engine in CI for releases and by developers locally resolving issues caused by different environment setups.

The rest of this document will focus on how to use the scripts locally, with an implementation overview at the end.

Requirements

You need to know the basics of how to use command line, git, and have the engine source checked out locally.

Windows

You can use the scripts natively from Windows or use Windows Subsystem for Linux (WSL).

Native Setup

  1. Install Docker Desktop following official instructions.
  2. Install a Bash shell, you have two options:

When executing commands in the Usage section, make sure that Docker is running, and execute all commands from the installed Bash command line.

Warning

This setup is very slow in comparison to other ones. When a compiler running inside a container accesses source files it crosses the boundary between the virtual machine and Windows host system which has a huge performance penalty (Some parts of compilation, e.g. configuration, can be even 100x slower).

WSL Setup

  1. Set up WSL following official documentation tutorial. WSL 2 is required.
  2. You can install Docker Desktop on Windows following official instructions, or set up the container runtime inside of the WSL following Linux Setup instructions.

Important

Make sure you have the engine source checked out inside of the WSL file system, not directly on the Windows disk, before proceeding to the Usage section. There is a large overhead when accessing files outside of WSL especially for workflows like compilation. If source code is checked out outside of WSL, the overhead will be similar to the "Native Setup" described in the previous section.

Tip

If you're using VSCode you can use Remote Development Extension to conveniently work on files inside of the WSL drive.

Linux Setup

Ensure you have some container runtime installed. The scripts support any of the following:

  • Podman: will be available directly in your distribution repository, daemonless, and by default more secure.

  • Docker Engine: install following official instructions and don't skip the post-installation steps. The scripts also support the more secure Rootless mode of operation you can set up instead.

  • Docker Desktop: install following official instructions. Note that Docker Desktop on Linux runs a virtual machine which has a performance overhead in comparison to Podman and Docker Engine which run natively in containers on the host.

Note

When both Podman and Docker are installed, by default the script chooses to use Docker. It can be overridden via the CONTAINER_RUNTIME environment variable.

Usage

After installing requirements, you can execute the build locally using the build.sh script from a shell:

$ docker-build-v2/build.sh --help
Usage: docker-build-v2/build.sh [-h|--help] [--configure|--compile] [-j|--jobs {number_of_jobs}] [--arch {arm64|amd64}] {windows|linux} [cmake_flag...]
Options:
  -h, --help   print this help message
  --configure  only configure, don't compile
  --compile    only compile, don't configure
  -j, --jobs   number of concurrent processes to use when building
  --arch       arm64 or amd64, defaults to host

Some behaviors can be changed by setting environment variables. Consult the script source for those more advanced use cases.

For example

docker-build-v2/build.sh windows

will:

  1. Automatically fetch the correct Docker image with the engine build environment from GitHub packages
  2. Configure the release configuration of the engine build
  3. Compile and install the engine using the following paths in the repository root:
    • .cache: compilation cache
    • build-amd64-windows: compilation output
    • build-amd64-windows/install: ready to use installation

Caution

The build output like in archives fetched from the releases page is inside of build-amd64-windows/install directory, not build-amd64-windows. The improvement is tracked in #2742.

TODO: Link to documentation article about how to start engine, load game in it etc once it exists. Some current references of not best quality specifically for BAR:

Tip

Don't forget that symlinks exist and can be used to link compilation output build-amd64-windows/install to the game installation folder. It can be helpful if your Recoil game/lobby supports starting arbitrary engine versions. For example, skylobby for generic lobby software, and for BAR, there is the Debug Launcher.

Custom build config

The script accepts CMake arguments, so the compilation can be easily customized. For example to compile Linux release with Tracy support and skip building headless:

docker-build-v2/build.sh linux -DBUILD_spring-headless=OFF -DTRACY_ENABLE=ON

To list all cmake options and their values run:

docker-build-v2/build.sh --configure linux -LH

When the configuration phase is not selected the arguments are passed to the compilation phase, so it can run custom targets (with options).

For example, to compile the unit tests with verbose output run:

docker-build-v2/build.sh --compile linux -t tests --verbose

In the case both configuration and compilation phase are selected, the configuration phase takes precedent over the arguments.

Custom Docker image

The official images are built as part of a CI workflow (see Implementation Overview), but if you want to adjust the Docker build image or test local changes, you can build it locally:

docker-build-v2/images/amd64-windows/build.sh

and then pass the custom image to the build script via the environment variable CONTAINER_IMAGE:

CONTAINER_IMAGE=recoil-build-amd64-windows docker-build-v2/build.sh windows

and build.sh will use it.

For details on private testing, check the wiki here

Implementation Overview

There are multiple separate build images, for Windows, for Linux, and each supported architecture. The Docker images are built as part of a GitHub Actions workflow and stored in the GitHub Package repository. The images are relatively small (~300-400MiB compressed) and building them takes 2-3 minutes.

Each of the images contains a complete required build environment with all dependencies installed (including mingwlibs etc.), configured for proper resolution from engine CMake configuration, and caching with ccache.

The build step is then a platform agnostic invocation of CMake with generic release build configuration.

To sum up, there is a separation:

  • Build environment: dependencies, toolchain, etc. are part of the Docker image.
  • Build options: e.g. optimization level, are in the platform agnostic build script docker-build-v2/scripts/configure.sh stored in the repository.
  • Post build scripts: platform agnostic scripts:
    • docker-build-v2/scripts/split-debug-info.sh: Splits debug information from binaries
    • docker-build-v2/scripts/package.sh: Creates 2 archives, one with engine and one with debug symbols.