On 02/25/25 at 02:02am, Kairui Song wrote: > From: Kairui Song <kasong@xxxxxxxxxxx> > > Current allocation workflow first traverses the plist with a global lock > held, after choosing a device, it uses the percpu cluster on that swap > device. This commit moves the percpu cluster variable out of being tied > to individual swap devices, making it a global percpu variable, and will > be used directly for allocation as a fast path. > > The global percpu cluster variable will never point to a HDD device, and > allocations on a HDD device are still globally serialized. > > This improves the allocator performance and prepares for removal of the > slot cache in later commits. There shouldn't be much observable behavior > change, except one thing: this changes how swap device allocation > rotation works. > > Currently, each allocation will rotate the plist, and because of the > existence of slot cache (one order 0 allocation usually returns 64 > entries), swap devices of the same priority are rotated for every 64 > order 0 entries consumed. High order allocations are different, they > will bypass the slot cache, and so swap device is rotated for every > 16K, 32K, or up to 2M allocation. > > The rotation rule was never clearly defined or documented, it was changed > several times without mentioning. > > After this commit, and once slot cache is gone in later commits, swap > device rotation will happen for every consumed cluster. Ideally non-HDD > devices will be rotated if 2M space has been consumed for each order. > Fragmented clusters will rotate the device faster, which seems OK. > HDD devices is rotated for every allocation regardless of the allocation > order, which should be OK too and trivial. > > This commit also slightly changes allocation behaviour for slot cache. > The new added cluster allocation fast path may allocate entries from > different device to the slot cache, this is not observable from user > space, only impact performance very slightly, and slot cache will be > just gone in next commit, so this can be ignored. > > Signed-off-by: Kairui Song <kasong@xxxxxxxxxxx> > --- > include/linux/swap.h | 11 ++-- > mm/swapfile.c | 136 +++++++++++++++++++++++++++++-------------- > 2 files changed, 95 insertions(+), 52 deletions(-) Reviewed-by: Baoquan He <bhe@xxxxxxxxxx>
