Skip to content

fix(query): preserve scalar bloom hashes in replace into - #20141

Merged
dantengsky merged 1 commit into
databendlabs:mainfrom
dantengsky:codex/fix-replace-scalar-bloom
Jul 14, 2026
Merged

fix(query): preserve scalar bloom hashes in replace into#20141
dantengsky merged 1 commit into
databendlabs:mainfrom
dantengsky:codex/fix-replace-scalar-bloom

Conversation

@dantengsky

@dantengsky dantengsky commented Jul 13, 2026

Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Distributed REPLACE INTO can panic when an external Parquet source projects a constant conflict
key:

panicked at deletion_accumulator.rs:45:25:
assertion `left == right` failed
  left: 3
 right: 4

The SELECT-list constant remains a Scalar in a locally processed source block, but remote exchange
materializes it as a Column. The previous Bloom hashing code silently skipped the Scalar, so deletion
information for the same target block contained three Bloom hash columns from a local block and four
from a remote block, triggering the assertion.

This PR materializes every selected Bloom key as a full Column before hashing. This prevents the
panic and keeps source Bloom hashes aligned with their target columns.

Changes

  • Replace filter_map(Value::as_column) with full-column materialization before Bloom hashing.
  • Add a unit regression covering Scalar and Column representations in one deletion accumulator.
  • Add a cluster-only regression with an external FROM 'fs:///...' Parquet source, a SELECT-list
    constant, an explicit three-node assertion, stable source/target partition placement, and an
    actual conflicting row replacement.

Verification

  • The exact cluster regression on unfixed debug build v1.2.925-nightly-1f0ac2a483 reproduces the
    left: 3, right: 4 panic.
  • With this fix on the same debug version, the exact cluster regression passes. The target row count
    is 23, and the conflicting row is replaced with the source value.
  • cargo fmt --all -- --check
  • git diff --check

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions Bot added the pr-bugfix this PR patches a bug in codebase label Jul 13, 2026
@dantengsky
dantengsky force-pushed the codex/fix-replace-scalar-bloom branch 16 times, most recently from 811c4d2 to 2cbb0ea Compare July 14, 2026 02:03
@dantengsky
dantengsky force-pushed the codex/fix-replace-scalar-bloom branch from 2cbb0ea to 4450123 Compare July 14, 2026 02:22
@dantengsky

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 4450123095

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@dantengsky
dantengsky requested review from SkyFan2002 and zhyass July 14, 2026 02:36
@dantengsky
dantengsky added this pull request to the merge queue Jul 14, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 14, 2026
@dantengsky
dantengsky added this pull request to the merge queue Jul 14, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 14, 2026
@dantengsky
dantengsky added this pull request to the merge queue Jul 14, 2026
Merged via the queue into databendlabs:main with commit 60f8aac Jul 14, 2026
94 checks passed
@dantengsky
dantengsky deleted the codex/fix-replace-scalar-bloom branch July 14, 2026 06:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants