Skip to content

fix(binder): correct join statistics propagation - #20122

Merged
zhang2014 merged 6 commits into
databendlabs:mainfrom
forsaken628:tpcds
Jul 27, 2026
Merged

fix(binder): correct join statistics propagation#20122
zhang2014 merged 6 commits into
databendlabs:mainfrom
forsaken628:tpcds

Conversation

@forsaken628

@forsaken628 forsaken628 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  • Correct join output statistics by handling join-key and non-key columns separately, using matched input rows to
    update NDV, null counts, and histograms according to join semantics
  • Fix DPhyp join reordering reusing stale pre-rewrite statistics by deriving statistics for each candidate and
    preserving the selected join tree’s updated estimates
  • Synchronize materialized CTE consumer estimates with the optimized producer so downstream optimizers do not use
    stale cardinalities
  • Add debug-time SExpr type consistency validation, including regression fixes for Decimal eager aggregation and
    aggregating-index local schema symbols
  • Add join-statistics unit tests, TPC-DS Q64 materialized-CTE optimizer replay coverage, and updated join-order
    goldens

Benchmark

tpcds sf 100 from 473.65s to 147.23s.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions Bot added the pr-bugfix this PR patches a bug in codebase label Jul 8, 2026
@forsaken628 forsaken628 added the ci-benchmark-local Benchmark: run only local test label Jul 9, 2026
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Docker Image for PR

  • tag: pr-20122-defd073-1783618683

note: this image tag is only available for internal use.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

@forsaken628
forsaken628 force-pushed the tpcds branch 2 times, most recently from cb7ad1c to adaf7bf Compare July 10, 2026 14:03
@forsaken628 forsaken628 added ci-benchmark-local Benchmark: run only local test and removed ci-benchmark-local Benchmark: run only local test labels Jul 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Docker Image for PR

  • tag: pr-20122-7e39c8a-1783741999

note: this image tag is only available for internal use.

@github-actions

Copy link
Copy Markdown
Contributor

@forsaken628 forsaken628 added ci-cloud Build docker image for cloud test and removed ci-benchmark-local Benchmark: run only local test labels Jul 14, 2026
@forsaken628 forsaken628 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Jul 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Docker Image for PR

  • tag: pr-20122-df32ed2-1784169690

note: this image tag is only available for internal use.

@forsaken628 forsaken628 changed the title fix(binder): cte fix(binder): correct join statistics propagation Jul 17, 2026
@forsaken628
forsaken628 marked this pull request as ready for review July 17, 2026 07:22

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9dbace494

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/query/sql/src/planner/optimizer/optimizers/hyper_dp/dphyp.rs
@forsaken628 forsaken628 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Docker Image for PR

  • tag: pr-20122-87bf6c8-1784516166

note: this image tag is only available for internal use.

@forsaken628
forsaken628 requested a review from zhang2014 July 20, 2026 03:25
@zhang2014
zhang2014 merged commit 2501da7 into databendlabs:main Jul 27, 2026
112 checks passed
@forsaken628
forsaken628 deleted the tpcds branch July 27, 2026 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants