Skip to content

[python] Avoid recursive copies of flat CWD vectors - #10627

Draft
YannByron wants to merge 1 commit into
apache:masterfrom
YannByron:codex/cwd-real-data-throughput
Draft

YannByron wants to merge 1 commit into
apache:masterfrom
YannByron:codex/cwd-real-data-throughput

Conversation

@YannByron

@YannByron YannByron commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

This PR remains draft: the long-window loader improvement is reproducible locally, but the target training-throughput improvement is not established.

CWD deep-copies each requested cell before applying transforms. Long action windows repeatedly recurse through the immutable scalars in numeric vectors. Copy flat lists of exact builtin immutable values directly; retain deepcopy for nested values, subclasses and custom objects so transforms and padding remain isolated.

This two-file follow-up to #10567 adds no sampler or cache configuration. The existing isolation test now checks overlapping/duplicate windows, repeated offsets, nested lists and mutable constant padding against explicit expected values.

Five alternating pairs on real RoboMIND JPEG data, after a discarded warmup round (samples/s; change is the ratio of medians):

Workload Base 220f0fe This change Change
Loader, horizon 32, 0 workers 170.39 170.13 -0.15%
Loader, horizon 128, 0 workers 142.04 150.71 +6.11%
Loader, horizon 32, 2 workers 307.06 309.23 +0.71%
CPU ACT, horizon 32, 0 workers (20 steps) 21.89 22.05 +0.73%
CPU ACT, horizon 32, 2 workers (20 steps) 26.02 25.47 -2.13%
CPU ACT, horizon 128, 0 workers (20 steps) 20.75 20.93 +0.85%
CPU ACT, horizon 32, 2 workers (200 steps) 24.97 25.30 +1.33%

The 128-step loader's paired gains were +2.9% to +9.4% (median +6.4%). The 32-step loader has no stable gain. CPU training results do not establish a general training speedup or the target of exceeding HDF5/LeRobot by 30%. The longer 200-step check's paired gains were -2.73% to +3.67% (median -1.67%); all short-run results are retained above.

Measurement scope: M2 Pro / 32 GiB, Python 3.13, Torch 2.10, PyArrow 19, Python reader, local files and uncontrolled OS cache. Loader: four complete episodes / 2,299 frames, three original 640x480 JPEG views, unchanged sample order and float32 tensors, batch 16, four warmup and 64 measured batches. ACT: two training episodes, batch 2, a reduced CPU model with the same 64x80 resize for all readers; 20 measured steps after four warmup updates, plus the 200-step check after ten warmup updates. These are engineering measurements, not GPU or convergence validation.

Tests

  • Real HDF5, Paimon and official LeRobot image readers produce identical sample IDs and tensors in an independent replay through the same worker/collate path. Every ACT run also agrees on normalization, per-step losses and initial/final model hashes within its configuration.
  • CWD dataset/plan, window transforms, ACT benchmark, torch read and multimodal video regression scopes: 151 passed. One local native-plan test fails identically on unmodified master because the older installed native wheel lacks Plan.stats; current-wheel coverage is delegated to CI.
  • The extended isolation test rejects both no-copy and nested shallow-copy mutations; the original deepcopy baseline passes.
  • Focused flake8 and git show --check pass. Independent copy-semantics and benchmark-methodology review completed.
  • CI Python 3.10-3.13 each fail only the same existing native-plan test: the released binding has no Plan.stats(). This is the master compatibility regression addressed by the still-open [python] Plan natively with pypaimon-rust releases that lack Plan.stats() #10620; that fix is not included here. CI is not yet green.

Copy flat lists of builtin immutable values without recursively copying
each scalar. Keep deepcopy for nested and custom values so overlapping
windows and padding remain isolated before transforms.

Co-Authored-By: Codex <noreply@anthropic.com>
AI-Model: gpt-6
Co-Authored-By: Codex <noreply@ai-tool.com>
AI-Contributed/Feature: 15/15
AI-Contributed/UT: 35/35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant