Skip to content

[Pipe] Fix historical TsFile schema recovery race - #18378

Open
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-18314-dev-1.3
Open

[Pipe] Fix historical TsFile schema recovery race#18378
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-18314-dev-1.3

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Description

Backport of #18314 to dev/1.3.

  • Wait for the complete historical schema stream before loading historical TsFiles when the source captures all required schema events.
  • Propagate the wait flag through sync, async, and air-gap transfers, and disable per-load schema auto-creation while waiting.
  • Keep the schema validation lock through execution and preserve the 1.3 V1 seal-request fallback when schema waiting is not required.

Verification

  • Spotless passed for node-commons, datanode, and the modified integration test.
  • Checkstyle, main compilation, and test compilation passed for node-commons and datanode.
  • 40 tests passed across PipeDataNodeTaskBuilderTest, IoTDBDataNodeReceiverTest, PipeDataNodeThriftRequestTest, LoadTsFileAnalyzerTest, and ActiveLoadPathHelperTest.

This PR has:

  • been self-reviewed.
  • added or updated unit tests for the changed paths.
  • updated the historical activation race integration test.

* [Pipe] Fix historical TsFile schema recovery race

* Clarify load schema creation and lock lifecycle
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant