Skip to content

Make the local Swift store work out of the box - #476

Merged
theresnotime merged 2 commits into
masterfrom
local-swift
Oct 27, 2026
Merged

theresnotime merged 2 commits into
masterfrom
local-swift

Conversation

@jsnshrmn

Copy link
Copy Markdown
Member

Description

Run a local Keystone + Swift as part of the dev stack, set up and kept in volumes so it works out of the box and lasts as long as the local database.

  • docker-compose.override.yml adds a swift service (CSCfi keystone-swift, pinned to 2026.9.0) and a one-shot swift-init that runs bin/swift_local_init.sh on every up. It creates the project, user and application credential named in .env, and points the Swift endpoints at the compose network
  • Objects and Keystone's database live in the swift_objects and swift_keystone volumes, so they survive docker compose down and go away with down -v, like the database
  • externallinks and crons wait for swift-init to finish
  • swift_connection() also accepts a credential name plus username when no ID is set. Keystone generates credential IDs, so template.env can't ship one. An ID still takes precedence, so production auth is unchanged
  • CI starts only the base compose file, since the tests mock Swift
  • The README's manual setup walkthrough is replaced with a short section, including how to migrate Keystone's database after bumping the image

Note: an existing .env needs the Swift block copied from template.env. Without it, swift-init fails and the web app and crons won't start.

Rationale

Production relies on Swift for link event and aggregate archives, but a local store meant a page of manual openstack commands from the README, redone whenever the container was recreated, since Keystone kept its state inside it.

Phabricator Ticket

T442038

How Has This Been Tested?

  • Added tests for connecting by credential name, and for rejecting a name without a username. The full suite passes (189 tests, 4 skipped)
  • Starting from a fresh .env copied from template.env, swift-init succeeded on the first run and again against existing state
  • From a Django shell, the extlinks.common.swift helpers created a container and uploaded, listed and downloaded an object; get_archive_list() reached Swift
  • The object was still there after docker compose down and up; down -v removed all three volumes
  • The README's keystone-manage db_sync steps run cleanly against the kept database

To try it: copy the Swift block from template.env into .env, run docker compose up -d, then check docker compose logs swift-init ends with "Local Swift ready".

Screenshots of your changes (if appropriate):

N/A

Types of changes

What types of changes does your code introduce? Add an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

The breaking change is local dev only: see the .env note above.

Setting up a local Keystone + Swift took a page of manual openstack
commands, redone whenever the container was recreated, since Keystone
keeps its state inside it. A one-shot service now does it on every
`docker compose up`.

Keystone generates application credential IDs, so template.env can't
ship one. swift_connection() now also accepts a credential name plus
username; an ID still takes precedence, so production auth is
unchanged.

Note: an existing .env needs the Swift block from template.env, or
swift-init exits on the missing variables.

Assisted-by: Claude Opus 5.5
Bug: T442038
Production relies on Swift now, so local archives should last as long
as the local database does. Keeping the objects alone isn't enough:
Swift files them under the project's account ID, which a fresh Keystone
regenerates, so Keystone's database gets a volume too.

A kept database can meet a newer image's schema, so the image is
pinned. The web app and crons wait for swift-init so neither runs
before the credential exists.

Assisted-by: Claude Opus 5.5
Bug: T442038
@theresnotime

Copy link
Copy Markdown
Member

Quick testing so far (threw away my local and brought it up using this):

  • docker compose up -d --build: all services start.
  • swift healthcheck: passes, so curl and /healthcheck exist in the image.
  • swift-init: exits 0 and prints "Local Swift ready".

@DHardy-WMF DHardy-WMF left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving the Swift setup, @jsnshrmn! Feel free to merge this patch @MusikAnimal once you confirm it's working in your local environment too.

@theresnotime
theresnotime merged commit 0825849 into master Oct 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants