Skip to content

Native collect_set(struct) aggregate crashes on spill emit #5239

Description

@hsiang-c

Describe the bug

Native collect_set / collect_list over a struct with a non-nullable nested field aborts in GroupedHashAggregateStream::emit during spill:

ArrowError: column types must match schema types,
  expected List(Struct("colA": Boolean, ...))
  but found  List(Struct("colA": non-null Boolean, ...))  at column index 1

Call Stack: GroupedHashAggregateStream::emit (row_hash.rs:1134) ← spill ← try_update_memory_reservation. Fires only on the spill path.

Dependencies versions: DataFusion 54.1.0 / arrow 58.4.0.

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:aggregationHash aggregates, aggregate expressionsbugSomething isn't workingpriority:highCrashes, panics, segfaults, major functional breakage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions