Skip to content

Add unit tests and fix build args being saved as env vars - #157

Merged
kylemclaren merged 3 commits into
fly-apps:mainfrom
alexleventer:test-coverage-pr
Oct 22, 2026
Merged

kylemclaren merged 3 commits into
fly-apps:mainfrom
alexleventer:test-coverage-pr

Conversation

@alexleventer

@alexleventer alexleventer commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Why

The only tests today are the Dockerfile snapshot tests in test/test.js, plus the Shopify tests from #154. They check that each fixture generates the expected files, but nothing covers the CLI flags that save to package.json, the fly.io setup in fly.js, or most of the generator's detection logic. Two of the bugs that test reveals (#144, and the build-args one fixed here) had been sitting unnoticed.

The bun snapshot tests (base: bun, base: cache-bun, frameworks: svelte-bun) also fail on any machine without bun installed, so npm test was red locally even when CI was green.

Changes

  • New unit tests, 223 passing plus the 10 skipped below, run by npm test next to the snapshot tests:
    • test/cli.test.js runs index.js itself. Covers saving and resetting flags, adding and removing packages, envs, build args, instruction files, mount secrets, and --skip/--force.
    • test/gdf.test.js covers framework and runtime detection, picking the package manager from lockfiles, Node version selection, Next.js/Astro/Prisma/Litestream handling, and --skip/--force on existing files. The interactive overwrite prompt (Y/n/a/q/d) is not covered yet.
    • test/fly.test.js covers app name and secrets parsing, swap, sqlite volumes, LiteFS/consul, Remix/Adonis secrets, Prisma release commands, GitHub Actions staging, and Shopify env/config.
    • test/helpers.js provides a Sandbox: a scratch app dir plus a bin dir of stub flyctl/shopify/bun/etc. PATH is limited to the stubs plus /usr/bin and /bin, so tools installed through Homebrew, nvm, etc. can't change results.
  • Bun snapshots: test/test.js puts a stub bun on PATH when bun isn't installed. The snapshots already mask the version, and CI still uses real bun.
  • test:capture now runs only test/test.js, so refreshing snapshots doesn't also run the new unit tests.
  • Bug fix: index.js saved options.vars[stage] (env vars) under dockerfile.args, so --arg-<stage> values were replaced by that stage's env vars in package.json and lost on the next run. One-word fix: options.args[stage]. package.json files written by earlier versions are not repaired: stray env vars under dockerfile.args have to be removed by hand (or with --arg-<stage> NAME:).

Known bugs left as it.skip

Each has a // BUG: comment with the cause. I kept them out of this PR so it stays reviewable:

  • --instructions / --instructions-deploy crash (Does --instructions-deploy not work? #144). Fixed in Fix crash with --instructions and --instructions-deploy #155; unskip once it lands.
  • --swap with MB/GB suffixes is off by 10x (1GB gives 95MB of swap).
  • --swap with no suffix (e.g. --swap 0) throws.
  • A swap_size_mb set by hand in fly.toml is removed on every run.
  • flyMakeVolume never creates missing sqlite volumes (it loops over the wrong map).
  • A Next.js app with no scripts in package.json throws.
  • Node and Next.js versions are compared as strings ("8" > "22", "9.5.0" >= "14.2"). 3 tests.
  • Prisma + sqlite without a .gitignore crashes with path is not defined in .dockerignore.ejs.

Testing

  • npm test: 352 passing, 10 pending, 0 failing, without bun installed. main gives 126 passing and 3 failing on the same machine.
  • The new saves build arguments test fails on main and passes with the fix.
  • eslint clean.

🤖 Generated with Claude Code

Run test/*.test.js alongside the Dockerfile snapshot tests, and stub bun
when it isn't installed so the bun snapshots pass locally.
@kylemclaren
kylemclaren merged commit a2af315 into fly-apps:main Oct 22, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants