Skip to content

Package plugin releases as flat shared libraries #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michalkomorsdn merged 7 commits into from
Oct 7, 2026
Merged

Package plugin releases as flat shared libraries #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e206f1a
Package release archives as flat shared libraries.
cursoragent Oct 7, 2026
4338d11
Install release artifacts with CMake instead of Python.
cursoragent Oct 7, 2026
a15deec
Include non-system runtime libraries in plugin packages.
cursoragent Oct 7, 2026
a69ad6c
Link plugins with the static MSVC CRT and static third-party libraries.
cursoragent Oct 7, 2026
d970504
Build Kafka zlib and zstd as static PIC libraries.
cursoragent Oct 7, 2026
795ae84
Stop Kafka from rewriting install(CODE) on macOS.
cursoragent Oct 7, 2026
5caf0b6
Document static release archives and plugin config examples.
cursoragent Oct 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 64 additions & 8 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ on:
- "kafka/**"
- "sqs/**"
- "pubsub/**"
- "cmake/**"
- ".github/workflows/release.yml"
pull_request:
branches: [ "main" ]
Expand All @@ -20,6 +21,7 @@ on:
- "kafka/**"
- "sqs/**"
- "pubsub/**"
- "cmake/**"
- ".github/workflows/release.yml"
workflow_dispatch:
inputs:
Expand Down Expand Up @@ -88,7 +90,7 @@ jobs:
echo "ARTIFACT_PATH=servicebus/build/ServiceBusPlugin" >> "$GITHUB_ENV"
echo "CONFIGURE_ARGS=-DCMAKE_TOOLCHAIN_FILE=${VCPKG_INSTALLATION_ROOT}/scripts/buildsystems/vcpkg.cmake" >> "$GITHUB_ENV"
if [ "${{ runner.os }}" = "Linux" ]; then
echo "INSTALL_PACKAGES=build-essential cmake git ninja-build pkg-config" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=build-essential cmake git ninja-build pkg-config patchelf" >> "$GITHUB_ENV"
elif [ "${{ runner.os }}" = "macOS" ]; then
echo "INSTALL_PACKAGES=cmake ninja pkg-config" >> "$GITHUB_ENV"
else
Expand All @@ -100,7 +102,7 @@ jobs:
echo "ARTIFACT_PATH=sqs/build/SqsPlugin" >> "$GITHUB_ENV"
echo "CONFIGURE_ARGS=-DCMAKE_TOOLCHAIN_FILE=${VCPKG_INSTALLATION_ROOT}/scripts/buildsystems/vcpkg.cmake" >> "$GITHUB_ENV"
if [ "${{ runner.os }}" = "Linux" ]; then
echo "INSTALL_PACKAGES=build-essential cmake git ninja-build pkg-config" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=build-essential cmake git ninja-build pkg-config patchelf" >> "$GITHUB_ENV"
elif [ "${{ runner.os }}" = "macOS" ]; then
echo "INSTALL_PACKAGES=cmake ninja pkg-config" >> "$GITHUB_ENV"
else
Expand All @@ -112,7 +114,7 @@ jobs:
echo "ARTIFACT_PATH=pubsub/build/PubSubPlugin" >> "$GITHUB_ENV"
echo "CONFIGURE_ARGS=-DCMAKE_TOOLCHAIN_FILE=${VCPKG_INSTALLATION_ROOT}/scripts/buildsystems/vcpkg.cmake" >> "$GITHUB_ENV"
if [ "${{ runner.os }}" = "Linux" ]; then
echo "INSTALL_PACKAGES=build-essential cmake git ninja-build pkg-config" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=build-essential cmake git ninja-build pkg-config patchelf" >> "$GITHUB_ENV"
elif [ "${{ runner.os }}" = "macOS" ]; then
echo "INSTALL_PACKAGES=cmake ninja pkg-config" >> "$GITHUB_ENV"
else
Expand All @@ -122,12 +124,15 @@ jobs:
kafka)
echo "DIRECTORY=kafka" >> "$GITHUB_ENV"
echo "ARTIFACT_PATH=kafka/build/KafkaPlugin" >> "$GITHUB_ENV"
echo "CONFIGURE_ARGS=" >> "$GITHUB_ENV"
if [ "${{ runner.os }}" = "Linux" ]; then
echo "INSTALL_PACKAGES=build-essential cmake git libssl-dev libsasl2-dev zlib1g-dev" >> "$GITHUB_ENV"
echo "CONFIGURE_ARGS=" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=build-essential cmake git libssl-dev libsasl2-dev patchelf" >> "$GITHUB_ENV"
elif [ "${{ runner.os }}" = "macOS" ]; then
echo "CONFIGURE_ARGS=" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=cmake openssl" >> "$GITHUB_ENV"
else
# Static OpenSSL/zlib built with /MT. The Program Files OpenSSL is /MD.
echo "CONFIGURE_ARGS=-DCMAKE_TOOLCHAIN_FILE=${VCPKG_INSTALLATION_ROOT}/scripts/buildsystems/vcpkg.cmake" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=cmake" >> "$GITHUB_ENV"
fi
;;
Expand All @@ -136,7 +141,7 @@ jobs:
echo "ARTIFACT_PATH=rabbitmq/build/RabbitmqPlugin" >> "$GITHUB_ENV"
echo "CONFIGURE_ARGS=" >> "$GITHUB_ENV"
if [ "${{ runner.os }}" = "Linux" ]; then
echo "INSTALL_PACKAGES=build-essential cmake git" >> "$GITHUB_ENV"
echo "INSTALL_PACKAGES=build-essential cmake git patchelf" >> "$GITHUB_ENV"
elif [ "${{ runner.os }}" = "macOS" ]; then
echo "INSTALL_PACKAGES=cmake" >> "$GITHUB_ENV"
else
Expand Down Expand Up @@ -173,10 +178,61 @@ jobs:
run: |
ctest --test-dir build -C Release --output-on-failure --timeout 180

# cmake --install --component plugin-runtime copies the plugin and the
# shared libraries it actually loads. FetchContent install rules and
# MSVC import libraries stay out of that component.
- name: Package (${{ matrix.plugin }})
shell: bash
run: |
mkdir -p release-payloads
tar -czf "release-payloads/${{ env.PLUGIN_NAME }}-${{ env.PACKAGE_SUFFIX }}.tar.gz" -C "${{ env.ARTIFACT_PATH }}" .
set -euo pipefail
staging="${GITHUB_WORKSPACE}/package-staging"
flat="${GITHUB_WORKSPACE}/package-flat"
rm -rf "${staging}" "${flat}"
mkdir -p "${staging}" "${flat}" release-payloads
cmake --install "${DIRECTORY}/build" --config Release --prefix "${staging}" --component plugin-runtime

find "${staging}" \( -type f -o -type l \) \( \
-name '*.dll' -o -name '*.DLL' -o \
-name '*.dylib' -o \
-name '*.so' -o -name '*.so.*' \
\) -print0 |
while IFS= read -r -d '' shared; do
cp -a "${shared}" "${flat}/$(basename "${shared}")"
done

count="$(find "${flat}" -maxdepth 1 \( -type f -o -type l \) | wc -l)"
if [[ "${count}" -eq 0 ]]; then
echo "error: no shared libraries installed from ${DIRECTORY}/build" >&2
find "${staging}" -print >&2 || true
exit 1
fi

stem="$(basename "${ARTIFACT_PATH}")"
plugin_found=0
for candidate in \
"${flat}/${stem}.dll" \
"${flat}/${stem}.DLL" \
"${flat}/lib${stem}.so" \
"${flat}/${stem}.so" \
"${flat}/lib${stem}.dylib" \
"${flat}/${stem}.dylib"
do
if [[ -e "${candidate}" ]]; then
plugin_found=1
fi
done
if compgen -G "${flat}/lib${stem}.so.*" >/dev/null || compgen -G "${flat}/${stem}.so.*" >/dev/null; then
plugin_found=1
fi
if [[ "${plugin_found}" -ne 1 ]]; then
echo "error: plugin shared library ${stem} was not installed" >&2
find "${flat}" -maxdepth 1 -print >&2
exit 1
fi

echo "Packaged shared libraries:"
find "${flat}" -maxdepth 1 \( -type f -o -type l \) -exec basename {} \; | sort
tar -czf "release-payloads/${PLUGIN_NAME}-${PACKAGE_SUFFIX}.tar.gz" -C "${flat}" .

- name: Upload artifact (${{ matrix.plugin }})
uses: actions/upload-artifact@v6
Expand Down
93 changes: 89 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,98 @@ OpenTelemetry / Application Insights lives under [observability/opentelemetry](o

Use RabbitMQ or Kafka when you already run that broker. Use Service Bus or SQS when the deployment is already on Azure or AWS. Use Pub/Sub when the deployment is already on Google Cloud. SQS and Pub/Sub each need one reply destination per client process; Service Bus can share one session-enabled reply queue.

## Release archives

Each release archive is flat and contains the plugin shared library. Third-party libraries are linked into that library. A client does not install OpenSSL, curl, zlib, the AWS SDK, the Azure SDK, or the VC++ Redistributable to load these plugins.

- **Windows.** The archive is the plugin DLL only. The MSVC CRT is static (`/MT`). `ucrtbase.dll` and other Windows system libraries stay on the machine.
- **Linux and macOS.** The archive is the plugin `.so` or `.dylib` only. glibc, libstdc++, libc++, and libSystem stay on the host.
- **Linux Kafka.** The archive also contains `libsasl2.so.2` and `libsasl2.so.2.0.25`. Cyrus SASL loads mechanism modules at runtime, so that library stays shared. Copy those files next to `libKafkaPlugin.so`.

`name` in the config is the library file name without the extension: `SqsPlugin` for `SqsPlugin.dll`, `libSqsPlugin` for `libSqsPlugin.so`.

## Use a plugin

1. Build the plugin (see its Readme). The artifact is a shared library: `SqsPlugin.dll` on Windows, or `libSqsPlugin.so` / `.dylib` on Linux and macOS.
2. Put `gg`, the hosted module, the plugin library, and any runtime DLLs in one folder. `name` in the config is the library file name without the extension (`libSqsPlugin` when the file is `libSqsPlugin.so`).
3. Start the Gateway with `--config`. A verified SQS layout is in [samples/sqs](samples/sqs/README.md): `gg netapp.dll --config sqsplugin.json`.
4. Open `http://localhost:81/GV`, install the generated package, and call `GraftConfig.SetConfig` with the same channel settings. `host` there is the Gateway HTTP address, not the broker.
1. Download the release archive for the platform, or build the plugin (see its Readme).
2. Put `gg`, the hosted module, and the plugin library in one folder. For Linux Kafka, also put `libsasl2.so.2` and `libsasl2.so.2.0.25` from the same archive there.
3. Start the Gateway with `--config` and one of the JSON files below. A verified SQS layout is in [samples/sqs](samples/sqs/README.md): `gg netapp.dll --config sqsplugin.json`.
4. Open `http://localhost:81/GV`, install the generated package, and call `GraftConfig.SetConfig` with the same channel settings inside `plugin`. `host` next to `runtime` is the Gateway HTTP address, not the broker.

How the Gateway loads a plugin is described in the "Plugin server config" section of the [Graftcode Gateway](https://github.com/grft-dev/graftcode-gateway) README.

## Gateway config

The Gateway `--config` file is the plugin object. The calling runtime wraps that same object in `configurations` → package name → `plugin`. Linux and macOS use a `lib` prefix on `name` (`libRabbitmqPlugin`). Field lists are in each plugin Readme.

RabbitMQ (`rabbitmq/Readme.md`):

```json
{
"name": "RabbitmqPlugin",
"host": "localhost",
"port": 5672,
"queue": "myqueue",
"replyQueue": "myqueue.reply",
"user": "guest",
"password": "guest",
"vhost": "/",
"rpcTimeoutMs": 30000
}
```

Service Bus (`servicebus/Readme.md`):

```json
{
"name": "ServiceBusPlugin",
"connectionString": "Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=<keyName>;SharedAccessKey=<key>",
"queue": "myqueue",
"replyQueue": "myqueue.reply",
"rpcTimeoutMs": 30000
}
```

Kafka (`kafka/Readme.md`):

```json
{
"name": "KafkaPlugin",
"brokers": "localhost:9092",
"requestTopic": "graft.requests",
"replyTopic": "graft.replies",
"groupId": "graft-gateway",
"rpcTimeoutMs": 30000
}
```

SQS (`sqs/Readme.md`, LocalStack):

```json
{
"name": "SqsPlugin",
"region": "us-east-1",
"endpointOverride": "http://localhost:4566",
"accessKeyId": "test",
"secretAccessKey": "test",
"requestQueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/graft-requests",
"waitTimeSeconds": 10,
"visibilityTimeoutSeconds": 60,
"verifySsl": false
}
```

Pub/Sub (`pubsub/Readme.md`, emulator):

```json
{
"name": "PubSubPlugin",
"projectId": "graftcode-local",
"emulatorHost": "localhost:8085",
"requestTopic": "graft-requests",
"ackDeadlineSeconds": 60,
"autoCreate": true,
"verifySsl": false
}
```

Part of the [Graftcode](https://github.com/grft-dev/graftcode) project.
Loading
Loading