Hi, Dave, Dave Hansen <dave.hansen@xxxxxxxxx> writes: > On 5/10/23 23:56, Huang Ying wrote: >> To improve the scalability of the page allocation, in this series, we >> will create one zone instance for each about 256 GB memory of a zone >> type generally. That is, one large zone type will be split into >> multiple zone instances. > > A few anecdotes for why I think _some_ people will like this: > > Some Intel hardware has a "RAM" caching mechanism. It either caches > DRAM in High-Bandwidth Memory or Persistent Memory in DRAM. This cache > is direct-mapped and can have lots of collisions. One way to prevent > collisions is to chop up the physical memory into cache-sized zones and > let users choose to allocate from one zone. That fixes the conflicts. > > Some other Intel hardware a ways to chop a NUMA node representing a > single socket into slices. Usually one slice gets a memory controller > and its closest cores. Intel calls these approaches Cluster on Die or > Sub-NUMA Clustering and users can select it from the BIOS. > > In both of these cases, users have reported scalability improvements. > We've gone as far as to suggest the socket-splitting options to folks > today who are hitting zone scalability issues on that hardware. > > That said, those _same_ users sometimes come back and say something > along the lines of: "So... we've got this app that allocates a big hunk > of memory. It's going slower than before." They're filling up one of > the chopped-up zones, hitting _some_ kind of undesirable reclaim > behavior and they want their humpty-dumpty zones put back together again > ... without hurting scalability. Some people will never be happy. :) Thanks a lot for your valuable input! > Anyway, _if_ you do this, you might also consider being able to > dynamically adjust a CPU's zonelists somehow. That would relieve > pressure on one zone for those uneven allocations. That wasn't an > option in the two cases above because users had ulterior motives for > sticking inside a single zone. But, in your case, the zones really do > have equivalent performance. Yes. For the requirements you mentioned above, we need a mechanism to adjust a CPU's zonelists dynamically. I will not implement that in this series. But I think that it's doable based on the multiple zone instances per zone type implementation in this series. Best Regards, Huang, Ying
