“The “optimized” code with one jump per loop is about 4x slower than the original version with two jumps per loop!”
Keeter removed one branch from an AArch64 inner loop and the sum went from 969 ns to 3.85 µs on an M1 Max. Reusing one bl for many ret instructions broke the return predictor, which then missed about 93% of the time. Swapping ret for br x30 brought it down to 913 ns. The CPU is guessing what your code means, and clever code makes it guess wrong.