Skip to content

[Bug] QDP streaming Parquet encoding copies an element count as a byte count #1463

Description

@viiccwen

What

The Linux QDP streaming Parquet path passes an f64 element count to PipelineContext::async_copy_to_device, whose third argument is a byte count. As a result, amplitude and angle encoders copy only one eighth of each input chunk to the GPU. The call succeeds, but the encoded state is silently computed from incomplete or stale staging-buffer data.

The issue affects encoders for which ChunkEncoder::needs_staging_copy() is true. Basis encoding is not affected because it validates and uploads indices through a separate path.

Steps to reproduce

  1. Create a Parquet file containing one or more non-zero angle samples. For example, use three qubits and the sample [0.1, 0.2, 0.3].

  2. Encode the file with:

    engine.encode_from_parquet(path, 3, "angle")
  3. Copy the returned DLPack state to the host.

  4. Compare each amplitude with the analytical angle-encoding result:

    psi[i] = product_k(bit_k(i) == 0 ? cos(x[k]) : sin(x[k]))
    

With the current implementation, the first amplitude is 1.0; the expected value for [0.1, 0.2, 0.3] is approximately 0.9316157966884513.

Root cause

ParquetBlockReader::read_chunk returns current_len as the number of f64 elements. PipelineContext::async_copy_to_device forwards its len_bytes argument directly to cudaMemcpyAsync. The streaming path currently passes current_len without multiplying it by size_of::<f64>().

Expected behavior

The full host chunk is copied to the device, and streaming amplitude/angle encoding produces the same numerical state as the corresponding analytical encoding.

Actual behavior

Only current_len bytes are copied instead of current_len * size_of::<f64>() bytes. Encoding succeeds with a numerically incorrect state.

Environment

  • OS: Linux
  • Python version: N/A (the defect is in the Rust qdp-core streaming path)
  • Qumat version: QDP 0.3.0-dev
  • Rust version: 1.85 or later
  • CUDA baseline: 12.5

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions