Repository navigation
[Engine] native_dense: STRATA_HC_Q8 uploads only the stage's own Q8_0 rows on a trimmed layer split - #3559
Open
havok2-htwo wants to merge 1 commit into
Open
[Engine] native_dense: STRATA_HC_Q8 uploads only the stage's own Q8_0 rows on a trimmed layer split#3559havok2-htwo wants to merge 1 commit into
havok2-htwo wants to merge 1 commit into
Conversation
…a trimmed layer split Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
bsorensen110
added a commit
to bsorensen110/Strata
that referenced
this pull request
Oct 22, 2026
…e's own Q8_0 rows (trimmed split)) — integ batch 19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Engine] native_dense: STRATA_HC_Q8 uploads only the stage's own Q8_0 rows on a trimmed layer split
Problem
With
--trim-stage-weightsandSTRATA_HC_Q8=1, the loop that attaches the GGUF's Q8_0 hyper-connection rows does not skip other stages' layers. Each trimmed card uploads all 48 layers' rows and reads only its own. Observed: 0.63 GiB per card with UD-IQ4_XS.Change
src/core/native_dense.cpp, +2 lines: the loop skips a tensor whenoutside(name) || !in_range(name), the filter the upload loop above it already uses. No value changes; without stage trimin_rangepasses every layer, so nothing changes.Validation
3d1fb480plus this commit (023b3962), CUDA sm_86 build with setup-style flags, 4x RTX 3090 24 GB.--layer-split 12,25,37 --trim-stage-weights,STRATA_HC_Q8=1, 16 slots, greedy.STRATA_HC_Q8=1: ... GiBline: stock 0.63 / 0.63 / 0.63 / 0.63; with this change 0.16 / 0.18 / 0.16 / 0.15.tools/batch_test.py --batch 16 --n 16 --max-new 128, same expert cache on every card): solo 16/16 and batch 16/16 identical between stock and this change.Not tested
HIP, Metal and SYCL builds (this file is compiled there, but not built here), 2- and 3-GPU splits,
--layer-split auto, long prompts, other models.