Puranjay Mohan <puranjay@xxxxxxxxxx> writes: > As ARM64 JIT now implements BPF_MOV64_PERCPU_REG instruction, inline > bpf_get_smp_processor_id(). > > ARM64 uses the per-cpu variable cpu_number to store the cpu id. While implementing this on the RISC-V JIT[1], I realized that reading the cpu_number from the per-cpu variable is not the best way to do this on arm64. arm64 now has the cpu number in the thread_info and reading that would be more efficient. Implementation in this patch is emitting: ; int cpu = bpf_get_smp_processor_id(); mov x7, #0xffff8000ffffffff movk x7, #0x8207, lsl #16 movk x7, #0x2008 mrs x10, tpidr_el1 add x7, x7, x10 ldr w7, [x7] If we do this in the JIT like I did for RISC-V[1] We can emit: ; int cpu = bpf_get_smp_processor_id(); mrs x10, sp_el0 ldr w7, [x10, #24] This gives ~ 10% improvement on glob-arr-inc and hash-inc compared to 3-4 % which this patch is providing. I will send v5 using this approach. [1] https://lore.kernel.org/all/20240430175834.33152-3-puranjay@xxxxxxxxxx/ > Here is how the BPF and ARM64 JITed assembly changes after this commit: > > BPF > ===== > BEFORE AFTER > -------- ------- > > int cpu = bpf_get_smp_processor_id(); int cpu = bpf_get_smp_processor_id(); > (85) call bpf_get_smp_processor_id#229032 (18) r0 = 0xffff800082072008 > (bf) r0 = &(void __percpu *)(r0) > (61) r0 = *(u32 *)(r0 +0) > > ARM64 JIT > =========== > > BEFORE AFTER > -------- ------- > > int cpu = bpf_get_smp_processor_id(); int cpu = bpf_get_smp_processor_id(); > mov x10, #0xfffffffffffff4d0 mov x7, #0xffff8000ffffffff > movk x10, #0x802b, lsl #16 movk x7, #0x8207, lsl #16 > movk x10, #0x8000, lsl #32 movk x7, #0x2008 > blr x10 mrs x10, tpidr_el1 > add x7, x0, #0x0 add x7, x7, x10 > ldr w7, [x7] > > Performance improvement using benchmark[1] > > BEFORE AFTER > -------- ------- > > glob-arr-inc : 23.817 ± 0.019M/s glob-arr-inc : 24.631 ± 0.027M/s [+ 3.41%] > arr-inc : 23.253 ± 0.019M/s arr-inc : 23.742 ± 0.023M/s [+ 2.10%] > hash-inc : 12.258 ± 0.010M/s hash-inc : 12.625 ± 0.004M/s [+ 3.00%] > > [1] https://github.com/anakryiko/linux/commit/8dec900975ef Thanks, Puranjay
