Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@ All notable changes to this project are documented in this file. The format is b

## [Unreleased]

### Added
- Every `smpy new` scaffold now ships Docker assets by default: a multi-stage
`docker/host.Dockerfile` (uv + Node builder that runs `gen-pages` before the
Vite build, slim non-root runtime that applies migrations on start), a
`docker-compose.yml` matched to the `--db` choice (`app` on a SQLite named
volume, or `postgres` + `app` — migration histories are dialect-frozen at
autogenerate time, so containers run the same DB the migrations were
generated against), plus `redis`/`worker`/`beat` reusing the app image when
`background_tasks` is selected, a `.dockerignore`, and `make docker-up` /
`docker-build` / `docker-down` targets. Previously Docker files only
appeared with `background_tasks`, and their frontend stage couldn't build
real apps (no `gen-pages` step). The separate `worker.Dockerfile` is gone;
worker/beat run the same image with a celery command. `smpy new` also
generates real `SM_USERS_*_TOKEN_SECRET` values into `.env.example` so the
production-mode containers pass `UsersSettings` boot validation.

### Fixed
- Vite's dev-mode dependency pre-bundling now resolves cross-package bare
imports (e.g. `maplibre-gl`, `pmtiles`) from module pages whose importers sit
Expand Down
4 changes: 2 additions & 2 deletions docs/guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,14 @@ For Postgres, scaffold with `--db postgres` (or pick Postgres in the wizard) and
SM_DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/myapp
```

A `docker-compose.yml` is only generated when you include the `background_tasks` module; it brings up `postgres` (db `simple_module`, user `sm`/`sm`) on `:5432`, `redis` on `:6379`, and the `host`/`worker`/`beat` services:
Every scaffold ships a `docker-compose.yml` that matches your `--db` choice (a migration history is dialect-frozen at autogenerate time, so containers must run the same database the migrations were generated against). With `--db postgres` it brings up `postgres` (db named after your app, user `postgres`/`postgres`) on `:5432` and the `app` service; the default SQLite scaffold runs the `app` container alone with the database on a named volume. Including the `background_tasks` module adds `redis` on `:6379` plus `worker`/`beat` services that reuse the app image. With `--db postgres`, the container also doubles as a local dev database:

```bash
docker compose up -d postgres
make migrate
```

Without `background_tasks`, bring your own Postgres (or stay on the default SQLite). See [Configuration](/guide/configuration) for the full list of env vars.
Or run the whole stack in containers — `make docker-up` builds the image (`docker/host.Dockerfile`), applies migrations on start, and serves on `:8000` with `SM_ENVIRONMENT=production`. See [Configuration](/guide/configuration) for the full list of env vars.

## Create the first admin

Expand Down
2 changes: 1 addition & 1 deletion docs/guide/project-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ myapp/
└── … (see "Anatomy of a module" below)
```

`smpy new --flat` skips the workspace wrapper and the sample `hello` module, emitting a single-host layout (`main.py`, `client_app/`, `migrations/` at the top level) — use it when the app only consumes published modules and never authors its own. `smpy new --preset minimal` ships fewer pre-installed modules. A `docker-compose.yml` is added only when the `background_tasks` module is included.
`smpy new --flat` skips the workspace wrapper and the sample `hello` module, emitting a single-host layout (`main.py`, `client_app/`, `migrations/` at the top level) — use it when the app only consumes published modules and never authors its own. `smpy new --preset minimal` ships fewer pre-installed modules. Every scaffold also ships Docker assets by default: a `docker/host.Dockerfile`, a `docker-compose.yml` matched to your `--db` choice (`app` on a SQLite volume, or `postgres` + `app`; plus `redis`/`worker`/`beat` when `background_tasks` is included), a `.dockerignore`, and `make docker-up` / `docker-build` / `docker-down` targets.

## Bundled modules

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Default Docker assets for `smpy new` — design

**Date:** 2026-08-06
**Status:** approved (autonomous session — assumptions listed below)

## Goal

Every application created by `smpy new` ships a working container story by
default: a `docker/host.Dockerfile`, a `docker-compose.yml`, a
`.dockerignore`, and `make docker-up` / `make docker-build` targets — for
every preset, both DB choices, and both layouts (workspace + flat).

Today Docker assets appear only when the `background_tasks` module is
selected, via its recipe. A default `smpy new myapp` (standard preset)
produces no Docker files at all. The optional templates are also broken for
real apps: the frontend stage never runs `gen-pages`, so the Vite build
fails on the missing `modules.generated.{ts,css}`, and `worker.Dockerfile`
copies `client_app/` from the root — a path that only exists in flat mode.

## Decisions

1. **Docker emission moves to an always-run scaffold step** —
`simple_module_cli/docker_assets.py`, called from `create_app_project`
after module recipes. It knows the resolved module selection, so it can
compose the right service set in one place. The `background_tasks`
recipe no longer writes compose/Dockerfiles (it keeps `run_worker.py`,
the broker env keys, and its Make targets).
2. **One image for app, worker, and beat.** The workspace venv already
contains everything (`uv sync --all-packages`), so worker/beat services
build the same `docker/host.Dockerfile` with a different `command:`.
`worker.Dockerfile` is deleted from the templates.
3. **Dockerfile follows the proven smpy_saas pattern** (the only known
working containerisation of a SimpleModule app): a
`ghcr.io/astral-sh/uv:python3.12-bookworm` builder stage with Node 22
installed runs `uv sync --all-packages` → `npm ci` → `smpy gen-pages` →
`vite build` → re-sync (so hatch force-include picks up `static/dist`);
a `python:3.12-slim-bookworm` runtime stage copies `.venv` + sources,
runs as a non-root user, healthchecks `GET /health`, and starts with
`alembic upgrade heads` (plural — singular errors once a second module
ships its own branch label) before uvicorn.
4. **Compose matches the scaffold's `--db` choice.** Migration histories
are dialect-frozen at autogenerate time (`sa.false()` compiles to
`DEFAULT 0` on SQLite, which Postgres rejects — found by booting a real
scaffold), so a sqlite scaffold's containers stay on SQLite (named
volume at `/app/data`) and only `--db postgres` scaffolds get a
`postgres` service (db/user matching `.env.example`:
`postgres`/`postgres`, db = kebab-case app slug). All app containers set
`SM_ENVIRONMENT=production` (a container has no Vite dev server;
development mode would emit asset tags pointing at `localhost:5050`).
Selecting `background_tasks` appends `redis`, `worker`, and `beat`
service fragments plus the `redisdata` volume — and injects the broker
URLs into the *app* service too, whose `BackgroundTasksSettings`
otherwise fails production boot on the localhost default.
5. **Compose is assembled from fragments** (per-DB `services` base +
optional per-DB tasks services + a computed `volumes:` block) rather
than merged YAML, keeping the templates dumb and the logic in one small
Python function. `smpy new` also generates real
`SM_USERS_RESET_PASSWORD_TOKEN_SECRET` /
`SM_USERS_VERIFICATION_TOKEN_SECRET` values into `.env.example` (like
`SM_SECRET_KEY`), because `UsersSettings` refuses its placeholder
secrets in production.
6. **Flat mode gets a flat variant Dockerfile** (`npm install` inside
`client_app/`, no `cd host`). Flat is legacy but should not silently
lose the default.

## Assumptions (would normally be clarifying questions)

- "Application" means the `smpy new` scaffold (not `make new-module`
modules, which are libraries and don't run standalone).
- Even sqlite-selected apps get the Postgres-backed compose: sqlite is the
zero-config *local dev* default, but a containerised deployment should
not write its database into an ephemeral container layer.
- The shared `~/Repos/dev-services` stack convention is a rule for this
workspace's own repos, not for scaffolded apps shipped to other users —
a self-contained compose is the correct default for the scaffold.

## Testing

- New `framework/cli/tests/test_cli_docker_assets.py`: default scaffold
emits compose/Dockerfile/dockerignore/Make targets; compose contains
`postgres` + `app` only; bg-tasks selection adds redis/worker/beat and
reuses `host.Dockerfile`; flat mode emits the flat variant; db name is
substituted.
- `test_cli_new.py` / `test_cli_recipes.py` updated: `worker.Dockerfile`
is gone; the recipe no longer owns compose.
- `docker compose config` validation of a scaffolded app's compose file
(cheap, no image pulls) as part of manual verification.
12 changes: 12 additions & 0 deletions framework/cli/simple_module_cli/app_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from simple_module_cli._env import set_env_key
from simple_module_cli.case import to_kebab_case, to_pascal_case
from simple_module_cli.catalog import CATALOG, PRESETS, expand_deps
from simple_module_cli.docker_assets import scaffold_docker_assets
from simple_module_cli.recipes import RECIPES, ScaffoldCtx
from simple_module_cli.scaffolding import (
SAFE_PRESERVED_NAMES,
Expand Down Expand Up @@ -141,6 +142,15 @@ def create_app_project(
env_path = target / ".env.example"
env_text = env_path.read_text(encoding="utf-8") if env_path.exists() else ""
env_text = set_env_key(env_text, "SM_SECRET_KEY", _secrets.token_urlsafe(32))
if "users" in resolved:
# UsersSettings refuses its placeholder token secrets when
# SM_ENVIRONMENT=production — the mode the compose containers run
# in — so generate real ones, same as SM_SECRET_KEY above.
for key in (
"SM_USERS_RESET_PASSWORD_TOKEN_SECRET",
"SM_USERS_VERIFICATION_TOKEN_SECRET",
):
env_text = set_env_key(env_text, key, _secrets.token_urlsafe(32))
env_text = set_env_key(env_text, "SM_DATABASE_URL", _db_url(db, to_kebab_case(name), flat=flat))
env_text = set_env_key(env_text, "SM_MULTI_TENANT", "true" if tenancy else "false")
env_path.write_text(env_text, encoding="utf-8")
Expand Down Expand Up @@ -176,6 +186,8 @@ def create_app_project(
if recipe_key is not None and recipe_key in RECIPES:
RECIPES[recipe_key].apply(target, ctx)

scaffold_docker_assets(target, ctx, flat=flat)

return host_dir, preserved


Expand Down
100 changes: 100 additions & 0 deletions framework/cli/simple_module_cli/docker_assets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
"""Default Docker assets for ``smpy new`` scaffolds.

Every new app ships a container story out of the box: a multi-stage
``docker/host.Dockerfile``, a ``docker-compose.yml`` matched to the
scaffold's DB choice (app on a SQLite volume, or postgres + app; plus
redis/worker/beat when ``background_tasks`` is selected), a
``.dockerignore``, and ``docker-*`` Make targets. This runs for every
scaffold — unlike per-module recipes — because the compose service set
depends on the *whole* module selection, which only the scaffolder knows.

worker/beat reuse the app image with a different command: the workspace
venv already contains everything (``uv sync --all-packages``), so a
separate worker Dockerfile would only duplicate build logic.
"""

from __future__ import annotations

import importlib.resources
import shutil
from pathlib import Path

from simple_module_cli.case import to_kebab_case
from simple_module_cli.recipes import ScaffoldCtx

__all__ = ["scaffold_docker_assets"]

_MAKEFILE_MARKER = "# --- docker --"
_PG_DB_TOKEN = "{{PG_DB}}"
_APP_EXTRA_ENV_TOKEN = "{{APP_EXTRA_ENV}}"

# BackgroundTasksSettings refuses a localhost broker when
# SM_ENVIRONMENT=production, so the app container needs the compose-network
# broker URLs too — not just worker/beat.
_APP_BROKER_ENV = (
" SM_BG_TASKS_BROKER_URL: redis://redis:6379/0\n"
" SM_BG_TASKS_RESULT_BACKEND: redis://redis:6379/1\n"
)


def _template_root() -> Path:
base = importlib.resources.files("simple_module_cli")
return Path(str(base / "templates" / "docker"))


def scaffold_docker_assets(target: Path, ctx: ScaffoldCtx, *, flat: bool = False) -> None:
"""Emit compose + Dockerfile + .dockerignore + Make targets at ``target``.

The compose file is assembled from fragments: a per-DB base (app alone
on SQLite, or postgres + app), per-DB worker services when
``background_tasks`` is selected, and a computed ``volumes:`` block —
appending fragments keeps the templates plain YAML instead of merge
logic.
"""
templates = _template_root()
compose_dest = target / "docker-compose.yml"
dockerfile_dest = target / "docker" / "host.Dockerfile"
for path in (compose_dest, dockerfile_dest):
if path.exists():
raise FileExistsError(
f"{path} already exists — refusing to clobber. "
"Remove the file or run `smpy new` against an empty directory."
)

dockerfile_dest.parent.mkdir(parents=True, exist_ok=True)
src_name = "host-flat.Dockerfile" if flat else "host.Dockerfile"
shutil.copy2(templates / src_name, dockerfile_dest)

# The compose stack must match the scaffold's DB choice: a migration
# history is dialect-frozen at autogenerate time (e.g. sa.false()
# compiles to DEFAULT 0 on SQLite, which Postgres rejects), so pointing
# a sqlite-scaffolded app at a Postgres container fails on first boot.
db = "postgres" if ctx.db == "postgres" else "sqlite"
pg_db = to_kebab_case(ctx.name)
with_tasks = "background_tasks" in ctx.selected
compose = (
_read(templates / f"compose-base-{db}.yml.tpl")
.replace(_PG_DB_TOKEN, pg_db)
.replace(_APP_EXTRA_ENV_TOKEN, _APP_BROKER_ENV if with_tasks else "")
)
volumes = ["pgdata"] if db == "postgres" else ["appdata"]
if with_tasks:
compose += _read(templates / f"compose-tasks-{db}.yml.tpl").replace(_PG_DB_TOKEN, pg_db)
volumes.append("redisdata")
compose += "\nvolumes:\n" + "".join(f" {name}:\n" for name in volumes)
compose_dest.write_text(compose, encoding="utf-8")

ignore_dest = target / ".dockerignore"
if not ignore_dest.exists():
shutil.copy2(templates / "dockerignore", ignore_dest)

makefile_path = target / "Makefile"
existing = makefile_path.read_text(encoding="utf-8") if makefile_path.exists() else ""
if _MAKEFILE_MARKER not in existing:
snippet = _read(templates / "Makefile.snippet")
sep = "" if existing.endswith("\n") or not existing else "\n"
makefile_path.write_text(existing + sep + snippet, encoding="utf-8")


def _read(path: Path) -> str:
return path.read_text(encoding="utf-8")
2 changes: 2 additions & 0 deletions framework/cli/simple_module_cli/new.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ def new_project(
typer.echo(' make migration msg="initial schema"')
typer.echo(" make migrate")
typer.echo(" make dev")
typer.echo(" make docker-up # or run the full stack in containers")
if "background_tasks" in resolved:
typer.echo(" docker compose up -d redis worker beat # background jobs")
return
Expand Down Expand Up @@ -168,6 +169,7 @@ def new_project(
# errors once a second module ships its own migration branch label.
subprocess.run([*_ALEMBIC, "upgrade", "heads"], cwd=host_dir, check=False)
typer.echo("\nSetup complete. Run `make dev` in the new directory.")
typer.echo("To run the full stack in containers instead: make docker-up")
if "background_tasks" in resolved:
typer.echo("For background jobs, also run: docker compose up -d redis worker beat")

Expand Down
33 changes: 11 additions & 22 deletions framework/cli/simple_module_cli/recipes.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,37 +49,26 @@ def _optional_template_root(name: str) -> Path:


class BackgroundTasksRecipe:
"""Lays down run_worker.py + compose + Dockerfiles + Make targets."""
"""Lays down run_worker.py + broker env keys + Make targets.

The compose services (redis/worker/beat) and the shared app image come
from the default Docker assets every scaffold gets — see
:mod:`simple_module_cli.docker_assets`.
"""

def apply(self, target: Path, ctx: ScaffoldCtx) -> None:
templates = _optional_template_root("background_tasks")

run_worker_dest = target / "scripts" / "run_worker.py"
compose_dest = target / "docker-compose.yml"
host_dockerfile_dest = target / "docker" / "host.Dockerfile"
worker_dockerfile_dest = target / "docker" / "worker.Dockerfile"

for path in (
run_worker_dest,
compose_dest,
host_dockerfile_dest,
worker_dockerfile_dest,
):
if path.exists():
raise FileExistsError(
f"{path} already exists — refusing to clobber. "
"Remove the file or run `smpy new` against an empty directory."
)
if run_worker_dest.exists():
raise FileExistsError(
f"{run_worker_dest} already exists — refusing to clobber. "
"Remove the file or run `smpy new` against an empty directory."
)

run_worker_dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(templates / "run_worker.py", run_worker_dest)

shutil.copy2(templates / "docker-compose.yml", compose_dest)

host_dockerfile_dest.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(templates / "host.Dockerfile", host_dockerfile_dest)
shutil.copy2(templates / "worker.Dockerfile", worker_dockerfile_dest)

env_path = target / ".env.example"
env_text = env_path.read_text(encoding="utf-8") if env_path.exists() else ""
env_path.write_text(
Expand Down
13 changes: 13 additions & 0 deletions framework/cli/simple_module_cli/templates/docker/Makefile.snippet
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# --- docker --------------------------------------------------------------
.PHONY: docker-build docker-up docker-down

docker-build: ## Build the app image
docker compose build

docker-up: ## Run the full stack in containers
@test -f .env || cp .env.example .env
docker compose up --build

docker-down: ## Stop the compose stack
docker compose down
# --- end docker ----------------------------------------------------------
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
services:
postgres:
image: postgres:16
environment:
POSTGRES_DB: "{{PG_DB}}"
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d {{PG_DB}}"]
interval: 5s
timeout: 5s
retries: 10

app:
build:
context: .
dockerfile: docker/host.Dockerfile
env_file:
- path: .env
required: false
environment:
# Containers serve the built bundle; development mode would emit
# asset tags pointing at the (absent) Vite dev server.
SM_ENVIRONMENT: production
SM_DATABASE_URL: postgresql+asyncpg://postgres:postgres@postgres:5432/{{PG_DB}}
{{APP_EXTRA_ENV}} ports:
- "8000:8000"
depends_on:
postgres:
condition: service_healthy
Loading
Loading