Skip to content

feat: add DDSketch - #314

Draft
tisonkun wants to merge 1 commit into
apache:mainfrom
tisonkun:codex/ddsketch
Draft

tisonkun wants to merge 1 commit into
apache:mainfrom
tisonkun:codex/ddsketch

Conversation

@tisonkun

Copy link
Copy Markdown
Member

Add ddsketch::DdSketch behind the ddsketch feature for mergeable numeric quantiles with relative value accuracy, useful for heavy-tailed latency distributions. The implementation follows the DDSketch paper, with Datadog's implementation and the C++ proposal as references.

Fixed logarithmic buckets use bounded dense stores for positive and negative values. The default collapses the smallest numerical values to preserve high-quantile accuracy; callers can select highest-value collapse instead. quantile_is_collapsed identifies estimates that have lost the configured accuracy. The API includes integer weighted updates, compatible merging, exact extrema, allocation reuse, and lower discrete quantiles without losing rank precision above 2^53.

Subnormal values share the zero bucket and have an absolute error bound. Serialization uses a versioned Rust-specific format and preserves collapse state; it does not claim Datadog or TCK wire compatibility. Documentation, a distributed latency example, behavior and malformed-image tests, and update/merge/query/serialization benchmarks are included.

Validation:

  • cargo x check, cargo x test, and cargo x lint.
  • DDSketch behavior and serialization tests on Rust 1.86.0.
  • ddsketch_latency example.
  • DDSketch Divan benchmarks with --sample-count 20 --min-time 0.05 --max-time 0.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant