On Thu, Mar 25, 2021 at 11:39:28AM -0300, Jason Gunthorpe wrote: > On Tue, Mar 23, 2021 at 09:27:38PM -0700, Aruna Ramakrishna wrote: > > > > Do you have benchmarks that show the performance of the high order > > > pages is not relavent? I'm a bit surprised to hear that > > > > > > > I guess my point was more to the effect that an order-8 alloc will > > fail more often than not, in this flow. For instance, when we were > > debugging the latency spikes here, this was the typical buddyinfo > > output on that system: > > > > Node 0, zone DMA 0 1 1 2 3 0 1 0 1 1 3 > > Node 0, zone DMA32 7 7 7 6 10 2 6 7 6 2 306 > > Node 0, zone Normal 3390 51354 17574 6556 1586 26 2 1 0 0 0 > > Node 1, zone Normal 11519 23315 23306 9738 73 2 0 1 0 0 0 > > > > I think this level of fragmentation is pretty normal on long running > > systems. Here, in the reg_mr flow, the first try (order-8) alloc > > will probably fail 9 times out of 10 (esp. after the addition of > > GFP_NORETRY flag), and then as fallback, the code tries to allocate > > a lower order, and if that too fails, it allocates a page. I think > > it makes sense to just avoid trying an order-8 alloc here. > > But a system like this won't get THPs either, so I'm not sure it is > relevant. The function was designed as it is to consume a "THP" if it > is available. So can we do this with just the addition of __GFP_NORETRY ? Jason
