Repository navigation
Give the default method table view one type for every stack depth - #1080
Merged
Merged
Conversation
The default `method_table_view` nested a `StackedMethodTable` per method table, and `GPUInterpreter` is parametric in the view, so back-ends stacking a different number of tables got different interpreter types. The inference code specialized on that type is only shared between packages whose stacks happen to have the same depth: GPUCompiler's own precompilation covers none of the back-ends, and Metal.jl precompiled ~15 s of CPU time faster only because KernelAbstractions' POCL back-end happened to stack as many tables, until KernelAbstractions 0.10.5 added one. Look up the tables in a flat `MethodTableStack` instead, with the same semantics as the nested chain, so every back-end that returns its tables from `method_tables` shares one interpreter type, and the inference code that GPUCompiler precompiles.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1080 +/- ##
==========================================
- Coverage 92.84% 92.81% -0.04%
==========================================
Files 31 31
Lines 7732 7775 +43
==========================================
+ Hits 7179 7216 +37
- Misses 553 559 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The default
method_table_viewnests aStackedMethodTableper method table, andGPUInterpreteris parametric in the view, so back-ends that stack a different number of tables get different interpreter types. The inference code specialized on that type is only shared between packages whose stacks happen to have the same depth. GPUCompiler's own precompilation workload (native target, one table) covers none of the back-ends.This showed up as a Metal.jl precompilation regression (32 → 38.7 s on CI, JuliaGPU/Metal.jl#1129). Metal stacks three tables plus
SHARED_METHOD_TABLE. Until KernelAbstractions 0.10.5, so did KA's POCL back-end, whose precompilation workload launches a kernel, so Metal reused the inference code in KA's package image. JuliaGPU/KernelAbstractions.jl#944 added a fourth table to POCL, and Metal had to compile it all again.This PR replaces the nested chain with a flat
MethodTableStackthat holds the tables in a vector. It has the same lookup semantics, so every back-end that returns its tables frommethod_tables(and the native target) shares one interpreter type, and with it the inference code GPUCompiler precompiles.Measured on an M1, Julia 1.13, user CPU time of
Base.compilecache(2 interleaved rounds):Compiling 29 new Metal kernels in a warm session takes the same time (1.23–1.32 s), so lookups through the flat stack don't slow down inference.
This doesn't change the back-end interface.
StackedMethodTableis still available, and back-ends that overridemethod_table_viewkeep working, but they get their own interpreter type and don't share the precompiled code. OpenCL.jl does that today and could switch tomethod_tables, though that would also addSHARED_METHOD_TABLEto its stack.