Repository navigation
BFloat16 tests fail on Apple M5 (ties not rounded to even) #949
Description
Two of the POCL BFloat16 tests (axpy and arithmetic) fail on my machine, an Apple M5 MacBook. CI's macOS-15, arm64 jobs pass, so this might be specific to newer Apple chips. Reproduced on main (5ff1632) after Pkg.update(), running only test/bfloat16.jl.
Test Summary: | Pass Fail Total Time
POCL BFloat16 | 18 2 20 2.5s
copy | 1 1 0.3s
axpy | 1 1 0.7s
arithmetic | 10 1 11 0.6s
...conversions | 7 7
Failures are at test/bfloat16.jl:74 (axpy) and :85 (a - b). The other arithmetic ops and all conversions pass.
I looked a bit further with a seeded copy of the test (4096 random values):
a - b: 15 elements differ, all by 1 ulp, and every one is an exact tie between two BFloat16 values. The host rounds those to even; the device result always has an odd mantissa.a * x + y: 332 of 4096 differ from the test's reference (up to 4 ulp).- Plain
Float32toBFloat16conversion on the device matches the host, including for exact ties.
So it looks like BFloat16 subtraction (and possibly the multiply/add) isn't rounding ties to even on this CPU, while the conversion itself is fine.
Test output
axpy: Test Failed at /Users/sasmith/Documents/gsocc/julia/KernelAbstractions.jl/test/bfloat16.jl:74
Expression: isequal(Array(out), map(((x, y)->begin
#= /Users/sasmith/Documents/gsocc/julia/KernelAbstractions.jl/test/bfloat16.jl:74 =#
BFloat16(Float32(BFloat16(Float32(a) * Float32(x))) + Float32(y))
end), hx, hy))
Evaluated: isequal(BFloat16[0.0, 0.0, 1.5, 0.0, -1.450711f25, -1.700052f24, -1.4584457f20, -3.2059688f-11, 0.0009765625, 1.5395671f-17 … 3.9549573f-24, -2.0921187f29, -2.5024764f26, 1.0364973f-16, 3.879011f32, 2.4702462f-15, -5.177344f6, 3104.0, -2.1018418f37, 7.6059036f30], BFloat16[0.0, 0.0, 1.5, 0.0, -1.450711f25, -1.700052f24, -1.4584457f20, -3.2059688f-11, 0.0009765625, 1.5504091f-17 … 3.9549573f-24, -2.0921187f29, -2.5024764f26, 1.0364973f-16, 3.879011f32, 2.4702462f-15, -5.177344f6, 3104.0, -2.1101494f37, 7.6059036f30])
arithmetic: Test Failed at /Users/sasmith/Documents/gsocc/julia/KernelAbstractions.jl/test/bfloat16.jl:85
Expression: isequal(o[2, :], map(f32(-), hx, hy))
Evaluated: isequal(BFloat16[-0.0, 0.0, 1.0, -5.0, -9.671407f24, 1.700052f24, -9.742187f19, 3.2059688f-11, 0.00065231323, 1.0299921f-17 … 2.6366382f-24, 2.0921187f29, 2.5024764f26, -1.0364973f-16, 2.5860072f32, 1.6514567f-15, -3.227648f6, -3104.0, -1.4039971f37, 5.0706024f30], BFloat16[-0.0, 0.0, 1.0, -5.0, -9.671407f24, 1.700052f24, -9.742187f19, 3.2059688f-11, 0.00065231323, 1.0299921f-17 … 2.6366382f-24, 2.0921187f29, 2.5024764f26, -1.0364973f-16, 2.5860072f32, 1.6514567f-15, -3.227648f6, -3104.0, -1.4039971f37, 5.0706024f30])
Test Summary: | Pass Fail Total Time
POCL BFloat16 | 18 2 20 2.5s
copy | 1 1 0.3s
axpy | 1 1 0.7s
arithmetic | 10 1 11 0.6s
Float32 to BFloat16 | 1 1 0.2s
Float64 to BFloat16 | 1 1 0.2s
Int32 to BFloat16 | 1 1 0.1s
Int64 to BFloat16 | 1 1 0.1s
UInt64 to BFloat16 | 1 1 0.1s
BFloat16 to Float32 | 1 1 0.1s
BFloat16 to Float64 | 1 1 0.1s
RNG of the outermost testset: Random.Xoshiro(0xaeeab5815a0957fc, 0xd42deffa4676f5e5, 0x3382e28cf033e54e, 0x6c7feaa896edc690, 0x24825cccfef22221)
versioninfo
Julia Version 1.13.3
Commit 67ac47b9cd8 (2026-10-25 15:28 UTC)
Build Info:
Official https://julialang.org release
Platform Info:
OS: macOS (arm64-apple-darwin27.0.0)
CPU: 10 × Apple M5
WORD_SIZE: 64
LLVM: libLLVM-20.1.8 (ORCJIT, apple-m1)
GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 10 virtual cores)
KernelAbstractions.jl version 0.10.5
Toolchain:
- Julia v1.13.3
- SPIRV_LLVM_Backend: 23.1.4+0
- SPIRV_Tools: 2025.4.0+0
- pocl_standalone: 7.2.3+4
Julia packages:
- GPUCompiler: 2.21.0
- LLVM: 10.4.3
- KernelInterface: 0.4.5
- SPIRVIntrinsics: 1.6.0
POCL Version:
OpenCL 3.0, PoCL 7.2~julia-a3a833749dc1 Apple, Release, RELOC, SPIR-V, LLVM 22.1.8jl, SLEEF, DISTRO, POCL_DEBUG
· cpu (fp16, fp64, il)
Activity
christiangnrd commentedon Nov 5, 2026
I can reproduce on M2. I suspect this wasn’t detected on M1 CI because those don’t have hardware bf16 support so both the pocl and the julia paths were emulated.
@maleadt is it sufficient to just replace the
isequalwithisapproxor do we want to leave it for when pocl gets native bf16 support?christiangnrd commentedon Nov 5, 2026
Claude thinks this will be fixed once we build pocl with llvm 23, that it was accidentally fixed by llvm/llvm-project#190164, and that we should leave the tests as-is.
sasmith-14 commentedon Nov 5, 2026
Makes sense, thanks for looking into it. Happy to re-run the bfloat16 tests on my M5 once there's a pocl build with llvm 23.
[pocl] Build the next JLLs with LLVM 23 (#15475)
[pocl] Fix LLVM 22's AArch64 BFloat16 rounding in the stable JLLs (#1…
sasmith-14 commentedon Nov 6, 2026
Tested on my M5: with the updated pocl and the tests from #952, the bfloat16 tests pass now (the
axpyone still failed with the previous pocl). With the tests onmain, they still fail, so it looks like both the pocl fix and #952 are needed. Thanks for fixing this so fast! (Claude helped me run the tests.)