On 10/30/24 3:01 PM, Jann Horn wrote: > On Wed, Oct 30, 2024 at 9:25?PM Jens Axboe <axboe@xxxxxxxxx> wrote: >> On 10/30/24 11:20 AM, Jann Horn wrote: >>> On Wed, Oct 30, 2024 at 5:58?PM Jens Axboe <axboe@xxxxxxxxx> wrote: >>>> This avoids array_index_nospec() for repeated lookups on the same node, >>>> which can be quite common (and costly). If a cached node is removed from >>> >>> You're saying array_index_nospec() can be quite costly - which >>> architecture is this on? Is this the cost of the compare+subtract+and >>> making the critical path longer? >> >> Tested this on arm64, in a vm to be specific. Let me try and generate >> some numbers/profiles on x86-64 as well. It's noticeable there as well, >> though not quite as bad as the below example. For arm64, with the patch, >> we get roughly 8.7% of the time spent getting a resource - without it's >> 66% of the time. This is just doing a microbenchmark, but it clearly >> shows that anything following the barrier on arm64 is very costly: >> >> 0.98 ? ldr x21, [x0, #96] >> ? ? tbnz w2, #1, b8 >> 1.04 ? ldr w1, [x21, #144] >> ? cmp w1, w19 >> ? ? b.ls a0 >> ? 30: mov w1, w1 >> ? sxtw x0, w19 >> ? cmp x0, x1 >> ? ngc x0, xzr >> ? csdb >> ? ldr x1, [x21, #160] >> ? and w19, w19, w0 >> 93.98 ? ldr x19, [x1, w19, sxtw #3] >> >> and accounts for most of that 66% of the total cost of the micro bench, >> even though it's doing a ton more stuff than simple getting this node >> via a lookup. > > Ah, actually... a difference between x86 and arm64 is that arm64 does > an extra Speculative Data Barrier here, while x86 just does some > arithmetic. Which I think is to work around "data value predictions", > in which case my idea of using bitwise AND probably isn't valid. > > https://developer.arm.com/documentation/102816/0205/ section "Software > Mitigations" says "Such code sequences are based around specific data > processing operations (for example conditional select or conditional > move) and a new barrier instruction (CSDB). The combination of both a > conditional select/conditional move and the new barrier are sufficient > to address this problem on ALL Arm implementations, both current and > future". Yep, see my followup on the x86-64 side too. Don't think it's worth doing something just because it's expensive on arm64, in fact any kind of higher frequency array_index_nospec() will be expensive on arm64 :/ -- Jens Axboe
