Repository navigation
test(integration): fix flaky JiT test racing on shared tmp dir cleanup - #2383
mbiernacik wants to merge 2 commits into
Conversation
| name: string, | ||
| fn: (projectDir: string, datasetName: string) => Promise<void>, | ||
| ) { | ||
| const projectDir = fs.mkdtempSync(path.join(process.env.TEST_TMPDIR, `${name}_`)); |
There was a problem hiding this comment.
nit: process.env.TEST_TMPDIR is only set under Bazel. Outside it, path.join(undefined, ...) throws. Maybe process.env.TEST_TMPDIR || os.tmpdir()?
There was a problem hiding this comment.
Good point, done.
|
Heads-up: we're going to close this PR in favour of #2385. Both PRs fix the same root cause: integration tests reusing fixed BigQuery dataset names, so concurrent runs and tests drop or overwrite each other's data. This PR does it only for The part that's unique to this PR, each JiT test owning and cleaning up its own temp directory instead of using |
The two tests in
tests/integration/jit.spec.tsrun in parallel and shared aTmpDirFixturecleaned up inafterEach, so the test that finished first deleted the other test's project directory. When that happened before the other test's JiT step, the JiT worker couldn't find@dataform/coreand the run failed withexpected 'FAILED' to deeply equal 'SUCCESSFUL'. BigQuery job history shows this in 16 of 388 CI runs over the last 90 days.Changes:
finally, so tests can't delete each other's files.