On Wed, 24 Jul 2024 at 13:33, Will Deacon <will@xxxxxxxxxx> wrote: > > On Tue, Jul 23, 2024 at 06:28:16PM +0200, Ard Biesheuvel wrote: > > On Tue, 23 Jul 2024 at 18:05, Will Deacon <will@xxxxxxxxxx> wrote: > > > > > > On Tue, Jul 23, 2024 at 05:02:15PM +0200, Ard Biesheuvel wrote: > > > > On Tue, 23 Jul 2024 at 16:52, Will Deacon <will@xxxxxxxxxx> wrote: > > > > > On Fri, Jul 19, 2024 at 11:02:29AM -0700, Ard Biesheuvel wrote: > > ... > > > > > > > > > > > > We might add > > > > > > > > > > > > if (pgtable_l4_enabled()) > > > > > > pgdp = &pgd; > > > > > > > > > > > > here to preserve the existing 'lockless' behavior when PUDs are not > > > > > > folded. > > > > > > > > > > The code still needs to be 'lockless' for the 5-level case, so I don't > > > > > think this is necessary. > > > > > > > > The 5-level case is never handled here. > > > > > > Urgh, yes, sorry. I've done a fantasticly bad job of explaining myself. > > > > > > > There is the 3-level case, where the runtime PUD folding needs the > > > > actual address in order to recalculate the descriptor address using > > > > the correct shift. In this case, we don't dereference the pointer > > > > anyway so the 'lockless' thing doesn't matter (afaict) > > > > > > > > In the 4-level case, we want to preserve the original behavior, where > > > > pgd is not reloaded from pgdp. Setting pgdp to &pgd achieves that. > > > > > > Right. What I'm trying to get at is the case where we have folding. For > > > example, with my patch applied, if we have 3 levels then the lockless > > > GUP walk looks like: > > > > > > > > > pgd_t pgd = READ_ONCE(*pgdp); > > > > > > p4dp = p4d_offset_lockless(pgdp, pgd, addr); > > > => Returns pgdp > > > p4d_t p4d = READ_ONCE(*p4dp); > > > > > > pudp = pud_offset_lockless(p4dp, p4d, addr); > > > => Returns &p4d, which is again the pgdp > > > pud_t pud = READ_ONCE(*pudp); > > > > > > > > > So here we're reloading the same pointer multiple times and my argument > > > is that if we need to add logic to avoid this for the > > > pgtable_l4_enabled() case, then we have bigger problems. > > > > > > > The 3-level case is not relevant here. My suggestion only affects the > > 4-level case: > > That's exactly what I'm uneasy about :/ > Right. > > if (pgtable_l4_enabled()) > > pgdp = &pgd; > > > > which prevents us from evaluating *pgdp twice, which seems to me to be > > the reason these routines exist in the first place. Given that the > > 3-level runtime-folded case is the one we are trying to fix here, I'd > > argue that keeping the 4-level case the same as before is important. > > I think consistency between 4-level and 3-level is far more important. > Adding code to avoid reloading the entry for one specific case (the > pgtable_l4_enabled() case), whilst requiring other cases (e.g. the > 3-level compile-time folded case) to reload from the pointer is > inconsistent. Either they both need it or neither of them need it, no? > The thing to keep in mind here is that the path via p4d_to_folded_pud() does not dereference the same pointer either. It just converts a p4d_t* to a pud_t* by deriving the page tables address from the p4d_t*, and applying the PUD_SHIFT rather than the P4D_shift which was applied one level up. So a) it does not dereference, and b) it refers to a different entry so the prior dereference loaded the wrong entry. OTOH, the 4-level path is not only used by 16k+lpa2 (or 16k/48bits), it is also used by 4k/48bits where pgtable_l4_enabled() is a compile-time constant TRUE, and this case will no longer be 'lockless' as before. However, if I am understanding you correctly, you are saying that a) p4d_offset_lockless() for <5 levels cannot race in the way that these macros are intended to address, as the folding implies that the next-level load is in reality a reload of the same entry, and so we will be using the latest value b) reloading the same value is not an issue because this is not a performance optimization but a concurrency/correctness thing. I suppose it would be good to clarify this in a comment, as doing the reload in the implementation of a helper that exists to omit it looks rather dodgy. But I agree with your analysis, and the additional bits I suggested are not needed.
