Skip to content

feat: publish a default and an -asr image on each release - #13

Draft
planetf1 wants to merge 11 commits into
mainfrom
feat/image-variants-and-build-args
Draft

planetf1 wants to merge 11 commits into
mainfrom
feat/image-variants-and-build-args

Conversation

@planetf1

@planetf1 planetf1 commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Until now the release image was amd64-only, decisions-only, and could not take speech input without extra setup. The audio libraries were missing, so a server started with speech on would pass /health and then fail every audio request with HTTP 500. After this PR each release publishes two multi-arch images (amd64 and arm64), the audio libraries are in both, and the README is written for people deploying the plugin rather than for us.

What changes for a user

  • ghcr.io/generative-computing/granite-switch-systemone:X.Y.Z (and latest) is the decisions image. X.Y.Z-asr (and latest-asr) also has the speech model baked in. Each tag is one manifest for linux/amd64 and linux/arm64.
  • With SYSTEMONE_SPEECH_INPUT=1, a missing audio library or speech model now stops the container at startup with a message that names the cause and the fix, instead of failing later.
  • The README is restructured: container, existing vLLM install and Hugging Face Inference Endpoints first, then request, security, speech, configuration, limits and test status. Everything in it was checked against the code.

How

  • image.yml builds each variant on a native runner per architecture (ubuntu-latest, ubuntu-24.04-arm), pushes by digest, and a second job joins the digests into the tags. A branch run pushes only <branch> and <branch>-asr.
  • Dockerfile installs granite-switch[audio] in every variant. ASR_REVISION is now the multilingual release of the speech model, pinned by commit; the earlier English-only revision is one build argument away.
  • preflight.py checks for av, scipy, soundfile and soxr as well as the speech model. scripts/speech_check.py is a standard-library check for speech input (clips are made at run time, none are committed).

Checked

  • Unit tests: 5 new preflight and serve.sh tests fail without the change and pass with it. actionlint, ruff and the SPDX header check pass.
  • Workflow dispatched from this branch: build-only run 40013172836 and push run 40014366911 both succeeded. The workflow's own imagetools inspect shows linux/amd64 and linux/arm64 in both tags.
  • Served from the pushed arm64 -asr image with docker run on an NVIDIA GB10 (public 8b, speech on): speech_check.py passed. The arm64 default image passed e2e_check, probe_check and accuracy_check, with accuracy matching amd64 on the same checkpoint.
  • speech_check.py also passed on amd64 (H100) with a mounted speech-model cache.
  • Not checked: the amd64 -asr image was not pulled and served, Hugging Face Inference Endpoints was not tried, and nothing was run under Docker on amd64.

For a reviewer

  • The ghcr package is private (an anonymous pull returns 401), so the README tells people to log in or build locally. Can an org admin make it public? If so, that sentence should be dropped before release.
  • The two branch tags feat-image-variants-and-build-args and feat-image-variants-and-build-args-asr are on ghcr and can be deleted after review.
  • Draft. I have not merged it.

The image workflow now builds two variants from a matrix: the default
(decisions only, no speech model) and an -asr variant built with
WITH_ASR=1 so chat speech input works without extra setup. Tags are
X.Y.Z and latest for the default, X.Y.Z-asr and latest-asr for the asr
image, so the two never push the same tag.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Describe GRANITE_SWITCH_VERSION, WITH_ASR, ASR_MODEL and ASR_REVISION,
and replace the 'no supported image is published' line with the ghcr
image name and tags. The package is currently private, so the README
says login is needed until it is made public.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
… startup

The default image lacked av, scipy, soundfile and soxr, so with
SYSTEMONE_SPEECH_INPUT=1 and a mounted speech model the server started and
passed /health, then answered every audio request with HTTP 500 "Please
install vllm[audio]". The image now installs granite-switch[audio] in every
variant (WITH_ASR=1 still only adds the baked-in speech model), and the
speech preflight names any missing audio library and stops before vllm
starts, for installs outside the image.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Move the default ASR_REVISION to the multilingual TurboCTC release (English,
French, German, Spanish, Portuguese, Italian and Japanese, with punctuation
and capitalization). The README says how to build with the earlier
English-only revision instead.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Reorder it so a reader gets a working server first: a quick start for the
container, an existing vLLM install and Hugging Face Inference Endpoints, a
first request, then securing it, speech input, configuration, behaviour and
limits. Remove restated internal detail, keep every tested fact and command,
and say plainly what has not been tried (Inference Endpoints).

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Each architecture builds on a native runner and pushes by digest. A second job
joins the digests into the version tag (and latest) for both the default and
the -asr image.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Standard library only. Sends spoken clips as input_audio and checks the reply
repeats most of the words. Clips are made at run time with espeak-ng or macOS
say, so no audio is stored here; --clip takes your own. Documented in
docs/TESTING.md, and the stale quick-start anchor there is fixed.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
…eech check

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Confirmed from the manifest lists a workflow push run produced for both the
default and -asr tags.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Pulled the pushed arm64 -asr image and served the public 8b with
SYSTEMONE_SPEECH_INPUT=1; scripts/speech_check.py passed.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Comments and docstrings only. The plugin's code is identical (checked by
comparing the AST with docstrings removed), the Dockerfile diff has no
non-comment lines, and the suite passes (226 passed, 4 skipped).

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant