Skip to content

foreach_range(...; isolate = true): run no other task while waiting - #87

Merged
maleadt merged 1 commit into
mainfrom
tb/isolate
Nov 5, 2026
Merged

maleadt merged 1 commit into
mainfrom
tb/isolate

Conversation

@maleadt

@maleadt maleadt commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Adds foreach_range(f, n; isolate = true): the caller dispatches and waits without switching tasks, provided f does not yield. The default and other loops are unchanged.

On Julia ≤ 1.13, yielding on thread 1 lets another task receive a Ctrl-C deferred by disable_sigint. A ThreadTeams worker can die before starting or drop the interrupt while idle (#40). This caused lost interrupts during KernelAbstractions CPU launches (JuliaGPU/KernelAbstractions.jl#941).

Isolated waits spin and park the OS thread; worker notifications use trylock + notify to avoid yielding on a contended event lock. If f yields and moves the caller onto another worker's thread, completion waits resume yielding so workers can progress.

Validation

  • Queued-task regression: fails with the yields restored; passes 160/160 under 16 CPU hogs.
  • Full suite passes on Julia 1.10 and 1.13 (-t 4).
  • KA SIGINT probe (kill -INT 0.25 s into a 2 s launch): previously lost 1 of 3 interrupts on Linux and the first in every run on macOS. With isolation, 12/12 (Linux) and 8/8 (macOS) delivered; all workers remained alive.

…ting

While it waits for the participants, the caller of a range region yields:
in the park branch of `rescue_ranges`, every 1024 spins of `wait_idle`,
and in `notify` when another task holds the event's lock. Tasks queued on
its thread then run in the middle of the call. With `isolate = true` the
caller only spins and parks its OS thread, and wakes workers after taking
the event's lock with `trylock`, so it switches tasks only where `f` does.
Should a call of `f` that yielded move the caller onto another worker's
thread, the waits yield again, since that worker could otherwise not
progress. The other loops and the default are unchanged.

A caller needs this when it relies on state of its thread across the call.
On Julia up to 1.13, `disable_sigint` defers a Ctrl-C only while the
deferring task runs on thread 1: when it yields, the interrupt is thrown
into whichever task runs there next, usually the thread-1 worker, which
dies (if it had not started yet) or drops it (#40). KernelAbstractions'
CPU back-end defers interrupts for the whole of a kernel launch on the
team, and lost them this way (JuliaGPU/KernelAbstractions.jl#941).
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Benchmark Results (Julia v1)

Time benchmarks
main 304ae40... main / 304ae40...
ad/forward tforeach team, 1000 4.09 ± 0.4 μs 8.51 ± 7.9 μs 0.48 ± 0.45
ad/forward tforeach team, 100000 0.0923 ± 0.25 ms 0.0979 ± 0.26 ms 0.943 ± 3.6
ad/reverse serial loop, 1000 9.03 ± 0.49 μs 9.03 ± 0.44 μs 1 ± 0.073
ad/reverse serial loop, 100000 0.727 ± 0.022 ms 0.726 ± 0.03 ms 1 ± 0.052
ad/reverse tforeach team, 1000 0.036 ± 0.0018 ms 0.0366 ± 0.0018 ms 0.984 ± 0.069
ad/reverse tforeach team, 100000 1.74 ± 0.94 ms 1.61 ± 1.2 ms 1.08 ± 1
ad/reverse tmapreduce team, 1000 21.3 ± 0.83 μs 24.4 ± 1.7 μs 0.873 ± 0.069
ad/reverse tmapreduce team, 100000 0.184 ± 0.029 ms 0.187 ± 0.42 ms 0.988 ± 2.2
fastbroadcast/9 arguments serial, 100x100x8 0.113 ± 0.013 ms 0.117 ± 0.016 ms 0.96 ± 0.17
fastbroadcast/9 arguments thread, 100x100x8 0.0869 ± 0.014 ms 0.0871 ± 0.019 ms 0.998 ± 0.28
fastbroadcast/log Base, 1000 6.65 ± 0.15 μs 6.64 ± 0.13 μs 1 ± 0.03
fastbroadcast/log Base, 100000 0.67 ± 0.01 ms 0.669 ± 0.011 ms 1 ± 0.022
fastbroadcast/log serial, 1000 6.7 ± 0.18 μs 6.7 ± 0.16 μs 1 ± 0.036
fastbroadcast/log serial, 100000 0.674 ± 0.012 ms 0.675 ± 0.013 ms 0.999 ± 0.026
fastbroadcast/log thread, 1000 3.56 ± 0.06 μs 3.65 ± 0.12 μs 0.975 ± 0.036
fastbroadcast/log thread, 100000 0.328 ± 0.013 ms 0.327 ± 0.013 ms 1 ± 0.056
overhead/@tasks team 0.5 ± 0.051 μs 0.49 ± 0.049 μs 1.02 ± 0.15
overhead/OhMyThreads tforeach 3.87 ± 0.27 μs 3.9 ± 0.28 μs 0.992 ± 0.099
overhead/Threads.@threads :dynamic 4.51 ± 1.3 μs 4.33 ± 1.4 μs 1.04 ± 0.44
overhead/serial loop 30 ± 10 ns 30 ± 10 ns 1 ± 0.47
overhead/tforeach greedy team 0.821 ± 0.08 μs 0.792 ± 0.07 μs 1.04 ± 0.14
overhead/tforeach team 0.49 ± 0.05 μs 0.481 ± 0.04 μs 1.02 ± 0.13
overhead/tmapreduce team 0.511 ± 0.041 μs 0.541 ± 0.05 μs 0.945 ± 0.12
trixi/reset du Threads.@threads :static 0.131 ± 0.0092 ms 0.13 ± 0.012 ms 1.01 ± 0.12
trixi/reset du serial 1.73 ± 0.19 μs 1.73 ± 0.19 μs 1 ± 0.16
trixi/reset du team 1.49 ± 0.14 μs 1.5 ± 0.14 μs 0.993 ± 0.13
trixi/reset du team, 10 in a row 13.2 ± 0.52 μs 13.2 ± 0.69 μs 0.995 ± 0.065
work/serial, 1000 3.15 ± 0.01 μs 3.15 ± 0.011 μs 1 ± 0.0047
work/serial, 100000 0.314 ± 0.01 ms 0.314 ± 0.01 ms 1 ± 0.046
work/tforeach team, 1000 2.02 ± 0.09 μs 2.09 ± 0.099 μs 0.967 ± 0.063
work/tforeach team, 100000 0.158 ± 0.013 ms 0.158 ± 0.0048 ms 1 ± 0.09
time_to_load 0.104 ± 0.0016 s 0.105 ± 0.0019 s 0.987 ± 0.024

@maleadt
maleadt merged commit bf58890 into main Nov 5, 2026
12 checks passed
@maleadt
maleadt deleted the tb/isolate branch November 5, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant