From: Kairui Song <kasong@xxxxxxxxxxx>
Slot cache was initially introduced by commit 67afa38e012e ("mm/swap:
add cache for swap slots allocation") to reduce the lock contention
of si->lock.
Previous series "mm, swap: rework of swap allocator locks" [1] removed
swap slot cache for freeing path as freeing path no longer touches
si->lock in most cased. Allocation path also have slight to none
contention on si->lock since that series, but slot cache still helps
to reduce other overheads, like counters and the plist.
This series removes the slot cache from allocation path too, by using
the cluster as allocation fast path and also reduce other overheads.
Now slot cache is completely gone, the code is much simplified without
obvious feature or performance change, also clean up related workaround.
Also this should avoid other potential issues, e.g. the long pinning
of swap slots: swap slot cache pins swap slots with HAS_CACHE, causing
reclaim or allocation fail to use these slots on scanning.
The only behavior change is the swap device allocation rotation
mechanism, as explained in the patch "mm, swap: use percpu cluster
as allocation fast path".
Test results are looking good after deleting the swap slot cache:
- vm-scalability with: `usemem --init-time -O -y -x -R -31 1G`,
12G memory cgroup using simulated pmem as SWAP (32G pmem, 32 CPUs),
16 test runs for each case, measuring the total throughput:
Before (KB/s) (stdev) After (KB/s) (stdev)
Random (4K): 424907.60 (24410.78) 414745.92 (34554.78)
Random (64K): 163308.82 (11635.72) 167314.50 (18434.99)
Sequential (4K, !-R): 6150056.79 (103205.90) 6321469.06 (115878.16)
- Build linux kernel with make -j96, using 4K folio with 1.5G memory
cgroup limit and 64K folio with 2G memory cgroup limit, on top of tmpfs,
12 test runs, measuring the system time:
Before (s) (stdev) After (s) (stdev)
make -j96 (4K): 6445.69 (61.95) 6408.80 (69.46)
make -j96 (64K): 6841.71 (409.04) 6437.99 (435.55)
The performance is unchanged, slightly better in some cases.
[1] https://lore.kernel.org/linux-mm/20250113175732.48099-1-ryncsn@xxxxxxxxx/
Kairui Song (7):
mm, swap: avoid reclaiming irrelevant swap cache
mm, swap: drop the flag TTRS_DIRECT
mm, swap: avoid redundant swap device pinning
mm, swap: don't update the counter up-front
mm, swap: use percpu cluster as allocation fast path
mm, swap: remove swap slot cache
mm, swap: simplify folio swap allocation
include/linux/swap.h | 21 +--
include/linux/swap_slots.h | 28 ----
mm/Makefile | 2 +-
mm/shmem.c | 21 +--
mm/swap.h | 6 -
mm/swap_slots.c | 295 ----------------------------------
mm/swap_state.c | 79 ++--------
mm/swapfile.c | 315 ++++++++++++++++++-------------------
mm/vmscan.c | 16 +-
mm/zswap.c | 6 +
10 files changed, 196 insertions(+), 593 deletions(-)
delete mode 100644 include/linux/swap_slots.h
delete mode 100644 mm/swap_slots.c
--
2.48.1
