Skip to content

feat(xio): make copy buffer size configurable - #11660

Merged
mudler merged 2 commits into
masterfrom
codex/configurable-copy-buffer
Aug 21, 2026
Merged

feat(xio): make copy buffer size configurable#11660
mudler merged 2 commits into
masterfrom
codex/configurable-copy-buffer

Conversation

@localai-bot

@localai-bot localai-bot commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds functional options to xio.Copy and a WithBufferSize option. Copies use a 1 MiB buffer by default while existing callers remain source compatible. Non-positive overrides fall back to the default.

This addresses severe progress-callback amplification during large model imports. The downloader reports progress after every copy chunk. In distributed mode, each report synchronously updates gallery_operations in PostgreSQL and publishes progress through NATS. The previous 32 KiB chunks can therefore turn one large import into hundreds of thousands of database updates and messages.

A live 10-second sample during a slow import recorded 250 NATS input messages and 244 PostgreSQL row updates. At 32 KiB per callback, 25 callbacks per second corresponds to 0.78 MiB/s, matching the observed import rate. A 1 MiB buffer reduces callback frequency by about 32x.

A separate production-shaped NAS probe showed similar raw SMB write throughput for both buffer sizes: 32 KiB averaged 71.7 MiB/s and 1 MiB averaged 72.3 MiB/s. This confirms that callback amplification, rather than CIFS write coalescing, is the bottleneck.

Notes for Reviewers

  • go test ./pkg/xio ./pkg/downloader ./pkg/oci
  • go test ./pkg/xio -run "^$" -bench BenchmarkCopy -benchtime=1x
  • Added Ginkgo coverage for complete copies, default and overridden sizes, invalid-size fallback, and cancellation.
  • Added benchmarks for the default, 32 KiB, 1 MiB, and 4 MiB buffers.

Signed commits

  • Yes, I signed my commits.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

mudler added 2 commits August 21, 2026 13:35
Document the context-aware copy buffer option and its validation plan.

Assisted-by: Codex:gpt-5
@mudler
mudler merged commit 5072219 into master Aug 21, 2026
69 of 70 checks passed
@mudler
mudler deleted the codex/configurable-copy-buffer branch August 21, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants