> On Apr 27, 2022, at 11:48 PM, Song Liu <songliubraving@xxxxxx> wrote: > > Hi Linus, > > Thanks for your thorough analysis of the situation, which make a lot of > sense. > >> On Apr 27, 2022, at 6:45 PM, Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> wrote: >> >> On Wed, Apr 27, 2022 at 3:24 PM Song Liu <songliubraving@xxxxxx> wrote: >>> >>> Could you please share your suggestions on this set? Shall we ship it >>> with 5.18? >> >> I'd personally prefer to just not do the prog_pack thing at all, since >> I don't think it was actually in a "ready to ship" state for this >> merge window, and the hugepage mapping protection games I'm still >> leery of. >> >> Yes, the hugepage protection things probably do work from what I saw >> when I looked through them, but that x86 vmalloc hugepage code was >> really designed for another use (non-refcounted device pages), so the >> fact that it all actually seems surprisingly ok certainly wasn't >> because the code was designed to do that new case. >> >> Does the prog_pack thing work with small pages? >> >> Yes. But that wasn't what it was designed for or its selling point, so >> it all is a bit suspect to me. > > prog_pack on small pages can also reduce the direct map fragmentation. > This is because libbpf uses tiny BPF programs to probe kernel features. > Before prog_pack, all these BPF programs can fragment the direct map. > For example, runqslower (tools/bpf/runqslower/) loads total 7 BPF programs > (3 actual programs and 4 tiny probe programs). All these programs may > cause direct map fragmentation. With prog_pack, OTOH, these BPF programs > would fit in a single page (or even share pages with other tools). Here are some performance data from our web service production benchmark, which is the biggest service in our fleet. We compare 3 kernels: nopack: no bpf_prog_pack; IOW, the same behavior as 5.17 4kpack: use bpf_prog_pack on 4kB pages (same as 5.18-rc5) 2mpack: use bpf_prog_pack on 2MB pages The benchmark measures system throughput under latency constraints. 4kpack provides 0.5% to 0.7% more throughput than nopack. 2mpack provides 0.6% to 0.9% more throughput than nopack. So the data has confirmed: 1. Direct map fragmentation has non-trivial impact on system performance; 2. While 2MB pages are preferred, bpf_prog_pack on 4kB pages also gives Significant performance improvements. Thanks, Song
