On 2021/8/18 16:57, Eric Dumazet wrote: > On Wed, Aug 18, 2021 at 5:33 AM Yunsheng Lin <linyunsheng@xxxxxxxxxx> wrote: >> >> This patchset adds the socket to netdev page frag recycling >> support based on the busy polling and page pool infrastructure. > > I really do not see how this can scale to thousands of sockets. > > tcp_mem[] defaults to ~ 9 % of physical memory. > > If you now run tests with thousands of sockets, their skbs will > consume Gigabytes > of memory on typical servers, now backed by order-0 pages (instead of > current order-3 pages) > So IOMMU costs will actually be much bigger. As the page allocator support bulk allocating now, see: https://elixir.bootlin.com/linux/latest/source/net/core/page_pool.c#L252 if the DMA also support batch mapping/unmapping, maybe having a small-sized page pool for thousands of sockets may not be a problem? Christoph Hellwig mentioned the batch DMA operation support in below thread: https://www.spinics.net/lists/netdev/msg666715.html if the batched DMA operation is supported, maybe having the page pool is mainly benefit the case of small number of socket? > > Are we planning to use Gigabyte sized page pools for NIC ? > > Have you tried instead to make TCP frags twice bigger ? Not yet. > This would require less IOMMU mappings. > (Note: This could require some mm help, since PAGE_ALLOC_COSTLY_ORDER > is currently 3, not 4) I am not familiar with mm yet, but I will take a look about that:) > > diff --git a/net/core/sock.c b/net/core/sock.c > index a3eea6e0b30a7d43793f567ffa526092c03e3546..6b66b51b61be9f198f6f1c4a3d81b57fa327986a > 100644 > --- a/net/core/sock.c > +++ b/net/core/sock.c > @@ -2560,7 +2560,7 @@ static void sk_leave_memory_pressure(struct sock *sk) > } > } > > -#define SKB_FRAG_PAGE_ORDER get_order(32768) > +#define SKB_FRAG_PAGE_ORDER get_order(65536) > DEFINE_STATIC_KEY_FALSE(net_high_order_alloc_disable_key); > > /** > > > >>
