On 1/16/25 00:15, Alexei Starovoitov wrote: > On Wed, Jan 15, 2025 at 3:47 AM Vlastimil Babka <vbabka@xxxxxxx> wrote: >> >> On 1/15/25 03:17, Alexei Starovoitov wrote: >> > From: Alexei Starovoitov <ast@xxxxxxxxxx> >> > >> > Introduce free_pages_nolock() that can free pages without taking locks. >> > It relies on trylock and can be called from any context. >> > Since spin_trylock() cannot be used in RT from hard IRQ or NMI >> > it uses lockless link list to stash the pages which will be freed >> > by subsequent free_pages() from good context. >> > >> > Do not use llist unconditionally. BPF maps continuously >> > allocate/free, so we cannot unconditionally delay the freeing to >> > llist. When the memory becomes free make it available to the >> > kernel and BPF users right away if possible, and fallback to >> > llist as the last resort. >> > >> > Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx> >> >> Acked-by: Vlastimil Babka <vbabka@xxxxxxx> >> >> With: >> >> > @@ -4853,6 +4905,17 @@ void __free_pages(struct page *page, unsigned int order) >> > } >> > EXPORT_SYMBOL(__free_pages); >> > >> > +/* >> > + * Can be called while holding raw_spin_lock or from IRQ and NMI, >> > + * but only for pages that came from try_alloc_pages(): >> > + * order <= 3, !folio, etc >> >> I think order > 3 is fine, as !pcp_allowed_order() case is handled too? > > try_alloc_page() has: > if (!pcp_allowed_order(order)) > return NULL; Ah ok I missed that the comment describes what pages try_alloc_pages() produces, not what's accepted here. > to make sure it tries pcp first. > bpf has no use for order > 1. Even 3 is overkill, > but it's kinda free to support order <= 3, so why not. > >> And >> what does "!folio" mean? > > That's what we discussed last year. > __free_pages() has all the extra stuff if (!head) and > support for dropping ref on the middle page. > !folio captures this more broadly. Aha! But in that case I realize we're actually wrong. It needs to be a folio (compound page) if it's order > 0 in order to drop that tricky !head freeing code. It's order > 0 pages that are not compound, that are problematic and should be eventually removed from the kernel. The solution is to add __GFP_COMP in try_alloc_pages_noprof(). This will have no effect on order-0 pages that BPF uses. Instead of "!folio" the comment could then say "compound".
