Repository navigation
Conversation
On x86-64 CPUs without native bfloat16 conversions, LLVM lowers fptrunc from float or double to bfloat to calls to __truncsfbf2 and __truncdfbf2. The JIT resolves them from the process, but libgcc only has them since GCC 13, and on Windows they can't be loaded, like the FP16 helpers. Define them for the JIT on x86-64, the way the FP16 helpers are on MinGW. They are written on the bits, returning a bfloat16 in the low half of a float's register, so building PoCL doesn't need __bf16 support. The double conversion rounds to float with round-to-odd first, so that it rounds only once.
SPV_KHR_bfloat16 adds a bfloat16 floating-point type to SPIR-V for loads, stores and conversions to and from other floating-point types. LLVM's SPIR-V back-end needs it for any LLVM IR that uses `bfloat`, e.g. from Julia's BFloat16, and the translator maps it back to `bfloat`. The SPIR-V reader rejects modules using extensions that are not enabled with --spirv-ext, so such modules failed to build. Enable the extension for the CPU devices when the translator reads it: libLLVMSPIRV is checked for it, since the 18-20 release branches only gained it in later point releases, and the llvm-spirv binary is assumed to read it from LLVM 21 on. Report it and the BFloat16TypeKHR capability, added to the vendored spirv.hh with the extension's other capabilities, through CL_DEVICE_SPIRV_EXTENSIONS_KHR and CL_DEVICE_SPIRV_CAPABILITIES_KHR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LLVM's SPIR-V back-end emits
SPV_KHR_bfloat16for any LLVM IR that usesbfloat, e.g. Julia'sBFloat16(Core.BFloat16since Julia 1.11, always lowered tobfloat). The CPU devices don't enable the extension for the translator, so such modules fail to build with "input SPIR-V module uses extension 'SPV_KHR_bfloat16' which were disabled by --spirv-ext option". This PR makes the CPU devices accept and report it.The extension adds a bfloat16 type for loads, stores and conversions. Arithmetic is not part of it; producers widen it to float.
+SPV_KHR_bfloat16is added toHOST_DEVICE_SPV_EXTENSIONSwhen the translator reads it. The translator has had it since 21.1.0, and since 18.1.14, 19.1.8 and 20.1.3 on the older release branches, so with libLLVMSPIRV, configure checks itsLLVMSPIRVExtensions.inc. A build that only has thellvm-spirvbinary, which can't be queried, gets the extension from LLVM 21 on. With LLVM 20 and an older translator library, it is left out.CL_DEVICE_SPIRV_EXTENSIONS_KHRlistsSPV_KHR_bfloat16andCL_DEVICE_SPIRV_CAPABILITIES_KHRlistsBFloat16TypeKHR, as is done forSPV_KHR_float_controls2. The vendoredspirv.hhgains the extension's three capabilities.__truncsfbf2and__truncdfbf2forfptrunctobfloat. libgcc has them only since GCC 13. On Windows they can't be loaded into the JIT, the same problem as with the FP16 helpers. The first commit defines them for the JIT on x86-64. They work on the bits and return the bfloat16 in the low half of a float register, so building PoCL doesn't need__bf16support. Converting from double rounds to float with round-to-odd first, so that the result is rounded only once. Compared with GCC 16's libgcc over 10^8 inputs (random bit patterns, values next to bfloat16 ties, overflow, NaN), they gave the same results for all.regression/spirv_bfloat16runs a SPIR-V kernel that converts floats to bfloat16 (ties, overflow, infinities, signed zeros) and bfloat16 to float, and takes a bfloat16 kernel argument. When the translator supports the extension, a CPU device that doesn't report it fails the test instead of skipping it.The translator deviates from the extension in two conversions, neither of which involves float:
OpFConvertbetween half and bfloat16 becomes anfptruncbetween two 16-bit types, which fails verification. LLVM IR can't express that conversion, so LLVM's SPIR-V back-end never emits it.OpFConvertfrom double to bfloat16 becomes a directfptrunc double to bfloat, which rounds once. The extension says to convert through binary32, which rounds twice. For SPIR-V produced from LLVM'sfptrunc double to bfloat, rounding once is what the source asked for.Tested with LLVM 22.1.8 and translator 22: ctest passes on main (344/344) and on 7.2 (354/354), including the new test with the native CPU target and with
POCL_LLVM_CPU_NAME=x86-64. The test fails when the new helper returns a wrong value, and when the extension is removed fromHOST_DEVICE_SPV_EXTENSIONS. End to end with KernelAbstractions.jl on the 7.2 backport, storing bfloat16, converting in both directions and an axpy (arithmetic widened to float by the producer) all give the same results as on the host.The 7.2 backport, as shipped in the Julia JLLs, is on
tb/spirv-bfloat16-7.2.