Repository navigation
Conversation
The thread that claims the last work-groups of a kernel unlinks it from the kernel queue, which takes wq_lock_fast. Until then, threads looking for work kept finding the kernel, took a reference, found no work-groups left and dropped the reference again, holding the lock each time. With many small launches this repeats for every kernel: launching 64 small work-groups back to back on 8 threads, up to 90% of the times a thread joined a kernel it found nothing to do. Skipping such kernels cuts those fruitless joins by 5x. remaining_wgs is now read without k->lock, so update it atomically, and keep it on its own cache line.
The work-groups of a kernel were dealt out in chunks to whichever thread asked first, so the same work-groups ran on different threads at every launch. On a NUMA system that rules out the usual way of placing memory: initializing it with a kernel, so that each page is first touched by, and thus placed near, the thread that later works on it. Pinning the threads with POCL_AFFINITY doesn't help. On 2x EPYC 7502 with 64 pinned threads, SAXPY over 4 GiB initialized by a kernel ran at 206 GB/s, where OpenMP-style static loops get 240. Split the work-groups into one contiguous block per thread instead, by linear index, as OpenMP's static schedule does. A thread claims half of what is left of its block at a time. A thread that runs out takes over the block of a thread that hasn't started on its own, or the back half of another block, runs half of that and makes the rest its block, where it can be taken again. Launches with as many work-groups thus mostly run the same work-groups on the same threads, while load imbalance is still evened out, as with LLVM OpenMP's static_steal schedule. The SAXPY above reaches 241 GB/s. Small launches cost about the same: on a Ryzen 9950X with 8 threads, a trivial kernel takes as long or less for 1 to 16384 work-groups (18.1 instead of 23.2 us for 16384), and with 16 threads at most 7% longer for 16 to 64 work-groups. A kernel whose work-items have very uneven costs runs as fast as before. POCL_PTHREAD_SCHEDULE=guided restores the previous schedule.
maleadt
added a commit
to JuliaPackaging/Yggdrasil
that referenced
this pull request
Oct 26, 2026
… driver PoCL's pthread driver dealt out work-groups to whichever thread asked first, so a kernel's work-groups ran on different threads at every launch, defeating NUMA first-touch placement. Give each thread a static block of work-groups with work stealing, stop threads from joining fully claimed kernels, and make POCL_AFFINITY pin threads to the allowed CPUs spread over NUMA nodes (JuliaGPU/pocl#7 and #8). Add them to the pocl and pocl_next series, and bump the standalone builds, whose bundled directories are symlinks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The pthread driver deals out the work-groups of a kernel in chunks to whichever thread asks first, so the same work-groups run on different threads at every launch. On a NUMA system that rules out the usual way to place memory, which is to initialize it with a kernel so that each page is first touched by the thread that later works on it.
POCL_AFFINITYdoesn't help with that. This was discussed in pocl#330 and pocl#1401.This PR makes the default schedule static, with stealing:
static_steal, without a fixed chunk size.POCL_PTHREAD_SCHEDULE=guidedrestores the previous schedule.The first commit fixes a convoy that the new schedule made worse. Until the thread that claimed a kernel's last work-groups unlinks the kernel, other threads kept joining it, found nothing to do, and held
wq_lock_fasteach time. Threads now skip kernels without unclaimed work-groups. That alone cuts such fruitless joins by 5x with the current schedule.Results
NUMA bandwidth. 2x EPYC 7502 (2 NUMA nodes), 64 threads pinned with
POCL_AFFINITY=1, SAXPY over 2x2 GiB initialized by a kernel: 206 GB/s before, 241 GB/s after. Pinned OpenMP-style static loops (Julia@threads :static) reach 240 GB/s.Small launches. Ryzen 9950X, a trivial kernel with 64 work-items per work-group, launched back to back and synchronized each time, both schedules alternated within one process (medians of 30 batches, µs per launch):
Small and medium launches cost about the same, larger ones less.
Uneven work. A kernel whose work-items' cost grows linearly with their index takes 79.6 ms with the new schedule and 79.2 ms with the old one.