fix(ai): expose TurboQuant padding on VectorQuantizeTask and correct its guidance

[sroussey]

Fixes on top of #354.

What breaks

packages/ai/src/task/VectorQuantizeTask.ts hard-rejected every non-power-of-2 dimensionality for method: "turbo", never exposed turboQuantizeToTypedArray's { padToPowerOf2: true }, and — the part that actually misleads — quoted RMSE figures for the cropped variant the util no longer emits, using them to steer users toward linear:

int8 cosine RMSE at d=768: linear 0.0027 vs turbo 0.0164

Cropping (keeping the first d of nextPowerOf2(d) rotated coordinates) really is worse than linear — that is why the util stopped offering it. But those numbers describe a code path that no longer exists, and the message presented them as the cost of turbo. So the task told users that linear was the accurate choice at 768/1536/3072 while the accurate choice, padding, was unreachable from the task at all.

Measured

Re-measured myself, 40 seeded pairs, int8, seed 42 — padded turbo vs this task's own linear path, RMSE of the quantized cosine against the exact cosine:

d	padded turbo	task linear	turbo advantage
768 (MiniLM)	0.00034	0.00269	7.8x
1536	0.00024	0.00331	13.8x
3072	0.00021	0.00263	12.8x

Padded turbo is roughly an order of magnitude more accurate at exactly the three sizes the old message named as reasons to avoid it. (Directionally identical to the figures in the brief; the multiples differ because the draws do.)

The fix

• New turboPadToPowerOf2 input, defaulting to false. Enabling it widens the output from d to nextPowerOf2(d). That is a real consequence — a storage column declared at d will reject the result — so it is an opt-in, never inferred. When set, the task passes { seed: turboSeed, padToPowerOf2: true } through to the util.
• Rejection message rewritten to lead with the flag, state the widening and the column sizing, quote the real measurement, and name linear second as the option that preserves length — which is the honest reason to pick it now.
• Same correction to the method description in inputSchema and to the util's JSDoc and throw. The cropped figures survive in the util's JSDoc, but explicitly labelled as the cropped variant's and marked as not to be quoted as the cost of turbo — they explain why the variant is not offered, which is still worth recording.
• Deduplicated nextPowerOf2. The util's copy calls assertDimensions first (rejecting non-integers, n < 1, n > 2**20); the task's local copy did neither. An oversized vector therefore bypassed the task's carefully worded error and surfaced the low-level one instead. The util's helper is now exported (with a comment on why callers must not reimplement it) and the local copy deleted.

What the new tests catch

All three were confirmed RED against the base branch before the fix, then green after:

• A 768-dim vector with turboPadToPowerOf2: true returns length 1024 (red today: unknown input property).
• Padded turbo beats method: "linear" on measured RMSE over seeded pairs at d=768. Asserted as turboRmse < linearRmse / 4 — measured gap is 7.8x, so there is headroom, but the direction is precisely what the old message got wrong and must not silently reverse.
• The existing rejection test now also asserts the message mentions turboPadToPowerOf2 and does not contain the stale 0.0164.

The comparison uses the file's seeded makeRandom, not Math.random(): it asserts a ratio between two quantizers, where an unlucky sample and a real regression would otherwise look the same.

Verification

Run in a worktree off this PR's base, bun install + bun run use-source:

• bun scripts/test.ts rag vitest — 24 files, 233 tests passed, 0 failed.
• bun run build:types — 40/41 tasks successful. The one failure is pre-existing on the base branch and unrelated to this PR: packages/test/src/test/util/TurboQuantize.test.ts imports getTestingLogger from ../../binding/TestingLogger, which does not exist. Confirmed by stashing this PR's changes and re-running: identical single error. It is fixed in the companion PR (fix(util): scale the TurboQuant norm, and repair three tests that could not fail #757), which also means that whole test file currently fails to load and none of its tests run. This PR is unaffected either way.
• prettier --check clean on all three changed files.

Environment note: Node 22.22.2, not the Node 24 CLAUDE.md asks for. Nothing here touches native deps, but CI on Node 24 is the authority.

Follow-ups, deliberately not in this PR

Both are additive API design that deserves its own review:

• A prepared-query API (turboPrepareQuery).
• A seed/method provenance tag on the stored record. Partially served today by the task's method / turboSeed outputs, but a padded vector is now a third thing a consumer may need to distinguish, since its length no longer matches the source embedding.

---

Generated by Claude Code