This is the short term solutiolns "swap cluster order" listed in my "Swap Abstraction" discussion slice 8 in the recent LSF/MM conference. When commit 845982eb264bc "mm: swap: allow storage of all mTHP orders" is introduced, it only allocates the mTHP swap entries from new empty cluster list. It has a fragmentation issue reported by Barry. https://lore.kernel.org/all/CAGsJ_4zAcJkuW016Cfi6wicRr8N9X+GJJhgMQdSMp+Ah+NSgNQ@xxxxxxxxxxxxxx/ The reason is that all the empty cluster has been exhausted while there are planty of free swap entries to in the cluster that is not 100% free. Remember the swap allocation order in the cluster. Keep track of the per order non full cluster list for later allocation. User impact: For users that allocate and free mix order mTHP swapping, It greatly improves the success rate of the mTHP swap allocation after the initial phase. Barry provides a test program to show the effect: https://lore.kernel.org/linux-mm/20240615084714.37499-1-21cnbao@xxxxxxxxx/ Without: $ mthp-swapout Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 2: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 4: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 5: swpout inc: 110, swpout fallback inc: 117, Fallback percentage: 51.54% Iteration 6: swpout inc: 0, swpout fallback inc: 230, Fallback percentage: 100.00% Iteration 7: swpout inc: 0, swpout fallback inc: 229, Fallback percentage: 100.00% Iteration 8: swpout inc: 0, swpout fallback inc: 223, Fallback percentage: 100.00% Iteration 9: swpout inc: 0, swpout fallback inc: 224, Fallback percentage: 100.00% Iteration 10: swpout inc: 0, swpout fallback inc: 216, Fallback percentage: 100.00% Iteration 11: swpout inc: 0, swpout fallback inc: 212, Fallback percentage: 100.00% Iteration 12: swpout inc: 0, swpout fallback inc: 224, Fallback percentage: 100.00% Iteration 13: swpout inc: 0, swpout fallback inc: 214, Fallback percentage: 100.00% $ mthp-swapout -s Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 2: swpout inc: 227, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 4: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 5: swpout inc: 33, swpout fallback inc: 197, Fallback percentage: 85.65% Iteration 6: swpout inc: 0, swpout fallback inc: 229, Fallback percentage: 100.00% Iteration 7: swpout inc: 0, swpout fallback inc: 223, Fallback percentage: 100.00% Iteration 8: swpout inc: 0, swpout fallback inc: 219, Fallback percentage: 100.00% Iteration 9: swpout inc: 0, swpout fallback inc: 212, Fallback percentage: 100.00% With: $ mthp-swapout Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 2: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 4: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 5: swpout inc: 227, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 6: swpout inc: 230, swpout fallback inc: 0, Fallback percentage: 0.00% ... Iteration 94: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 95: swpout inc: 221, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 96: swpout inc: 229, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 97: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 98: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 99: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 100: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00% $ mthp-swapout -s Iteration 1: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 2: swpout inc: 227, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 3: swpout inc: 222, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 4: swpout inc: 224, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 5: swpout inc: 230, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 6: swpout inc: 229, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 7: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 8: swpout inc: 219, swpout fallback inc: 0, Fallback percentage: 0.00% ... Iteration 94: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 95: swpout inc: 212, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 96: swpout inc: 220, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 97: swpout inc: 220, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 98: swpout inc: 216, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 99: swpout inc: 223, swpout fallback inc: 0, Fallback percentage: 0.00% Iteration 100: swpout inc: 225, swpout fallback inc: 0, Fallback percentage: 0.00% Reported-by: Barry Song <21cnbao@xxxxxxxxx> Signed-off-by: Chris Li <chrisl@xxxxxxxxxx> --- Changes in v3: - Using V1 as base. - Rename "next" to "list" for the list field, suggested by Ying. - Update comment for the locking rules for cluster fields and list, suggested by Ying. - Allocate from the nonfull list before attempting free list, suggested by Kairui. - Link to v2: https://lore.kernel.org/r/20240614-swap-allocator-v2-0-2a513b4a7f2f@xxxxxxxxxx Changes in v2: - Abandoned. - Link to v1: https://lore.kernel.org/r/20240524-swap-allocator-v1-0-47861b423b26@xxxxxxxxxx --- Chris Li (2): mm: swap: swap cluster switch to double link list mm: swap: mTHP allocate swap entries from nonfull list include/linux/swap.h | 30 +++---- mm/swapfile.c | 248 +++++++++++++++++---------------------------------- 2 files changed, 95 insertions(+), 183 deletions(-) --- base-commit: 19b8422c5bd56fb5e7085995801c6543a98bda1f change-id: 20240523-swap-allocator-1534c480ece4 Best regards, -- Chris Li <chrisl@xxxxxxxxxx>
