Skip to content

perf(docker): precompile Python bytecode in worker images - #114

Merged
deanq merged 1 commit into
mainfrom
deanq/sls-809-compile-bytecode
Oct 19, 2026
Merged

deanq merged 1 commit into
mainfrom
deanq/sls-809-compile-bytecode

Conversation

@deanq

@deanq deanq commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

uv pip install does not compile bytecode by default. The published runpod/flash-cpu:latest ships with zero .pyc for runpod and aiohttp, so every fresh container recompiles each imported module on first import, on every cold start.

  • --compile-bytecode on the build-time uv pip install in all four Dockerfiles
  • python -m compileall -q . for the worker source in /app
  • Deliberately a flag, not ENV UV_COMPILE_BYTECODE=1: an env var would leak into runtime installs in dependency_installer.py, where compiling whole packages at boot would add to cold start

Impact (arm64 CPU image, local builds, main vs this branch)

main this branch
.pyc runpod / aiohttp 0 / 0 80 / 55
Image size 177.1MB 187.9MB
Cold-container import runpod.serverless (6 fresh containers) 1177-1198ms 594-623ms

Image ships runpod 1.10.1; absolute numbers drop further once the worker moves to runpod 2.0 (lazy imports).

Test plan

  • Built Dockerfile-cpu locally; verified .pyc present and measured cold-container import before/after
  • GPU / LB images build in CI (amd64-only, not built locally; identical one-line change)
  • Confirm cold-start delta on a real amd64 worker

Note for local testing on Apple M4-class Macs: arm64 images SIGILL in cryptography (OpenSSL SME detection inside the Docker VM) on main as well; run with -e OPENSSL_armcap=0.

Linear: SLS-809 (related: SLS-808)

uv pip install does not compile bytecode by default, so the published
images ship with no .pyc for runpod, aiohttp, or the worker source.
Every fresh container recompiles each imported module on first import,
on every cold start.

- Pass --compile-bytecode to the build-time uv pip install in all four
  Dockerfiles (flag, not UV_COMPILE_BYTECODE env, so runtime installs
  in dependency_installer.py stay uncompiled)
- compileall the worker source in /app
- Cold-container import runpod.serverless: ~1180ms -> ~600ms
  (arm64 CPU image, local build); image +10.8MB

SLS-809

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The consistent, build-only changes preserve interpreter selection and runtime installation behavior, with no blocking issues identified.

No open issues 🎉

What changed in this PR

Reduces worker cold-start overhead by precompiling Python bytecode during image builds.

Changes:

  • Enables dependency bytecode compilation in all four images.
  • Precompiles worker source under /app.
  • Keeps compilation build-scoped without changing runtime dependency installs.
File Description
Dockerfile-lb-cpu Precompiles dependencies and source for CPU load-balanced workers.
Dockerfile-lb Precompiles dependencies and source for GPU load-balanced workers.
Dockerfile-cpu Precompiles dependencies and source for CPU queue-based workers.
Dockerfile Precompiles dependencies and source for GPU queue-based workers.

Review effort: Balanced

@deanq
deanq merged commit 4ddd481 into main Oct 19, 2026
26 checks passed
@deanq
deanq deleted the deanq/sls-809-compile-bytecode branch October 19, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants