The patch titled
Subject: Multi-gen LRU: fix workingset accounting
has been added to the -mm mm-unstable branch. Its filename is
mm-unstable-multi-gen-lru-fix-workingset-accounting.patch
This patch will shortly appear at
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-unstable-multi-gen-lru-fix-workingset-accounting.patch
This patch will later appear in the mm-unstable branch at
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days
------------------------------------------------------
From: Kalesh Singh <kaleshsingh@xxxxxxxxxx>
Subject: Multi-gen LRU: fix workingset accounting
Date: Tue, 23 May 2023 13:59:21 -0700
On Android app cycle workloads, MGLRU showed a significant reduction in
workingset refaults although pgpgin/pswpin remained relatively unchanged.
This indicated MGLRU may be undercounting workingset refaults.
This has impact on userspace programs, like Android's LMKD, that monitor
workingset refault statistics to detect thrashing.
It was found that refaults were only accounted if the MGLRU shadow entry
was for a recently evicted folio. However, recently evicted folios should
be accounted as workingset activation, and refaults should be accounted
regardless of recency.
Fix MGLRU's workingset refault and activation accounting to more closely
match that of the conventional active/inactive LRU.
Link: https://lkml.kernel.org/r/20230523205922.3852731-1-kaleshsingh@xxxxxxxxxx
Fixes: ac35a4902374 ("mm: multi-gen LRU: minimal implementation")
Signed-off-by: Kalesh Singh <kaleshsingh@xxxxxxxxxx>
Reported-by: Charan Teja Kalla <quic_charante@xxxxxxxxxxx>
Acked-by: Yu Zhao <yuzhao@xxxxxxxxxx>
Cc: Brian Geffon <bgeffon@xxxxxxxxxx>
Cc: Jan Alexander Steffens (heftig) <heftig@xxxxxxxxxxxxx>
Cc: Oleksandr Natalenko <oleksandr@xxxxxxxxxxxxxx>
Cc: Suren Baghdasaryan <surenb@xxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
mm/vmscan.c | 1 -
mm/workingset.c | 10 +++++++---
2 files changed, 7 insertions(+), 4 deletions(-)
--- a/mm/vmscan.c~mm-unstable-multi-gen-lru-fix-workingset-accounting
+++ a/mm/vmscan.c
@@ -4925,7 +4925,6 @@ static bool sort_folio(struct lruvec *lr
WRITE_ONCE(lrugen->protected[hist][type][tier - 1],
lrugen->protected[hist][type][tier - 1] + delta);
- __mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta);
return true;
}
--- a/mm/workingset.c~mm-unstable-multi-gen-lru-fix-workingset-accounting
+++ a/mm/workingset.c
@@ -278,6 +278,7 @@ static bool lru_gen_test_recent(void *sh
static void lru_gen_refault(struct folio *folio, void *shadow)
{
+ bool recent;
int hist, tier, refs;
bool workingset;
unsigned long token;
@@ -288,10 +289,13 @@ static void lru_gen_refault(struct folio
rcu_read_lock();
- if (!lru_gen_test_recent(shadow, type, &lruvec, &token, &workingset))
+ recent = lru_gen_test_recent(shadow, type, &lruvec, &token, &workingset);
+ if (lruvec != folio_lruvec(folio))
goto unlock;
- if (lruvec != folio_lruvec(folio))
+ mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta);
+
+ if (!recent)
goto unlock;
lrugen = &lruvec->lrugen;
@@ -302,7 +306,7 @@ static void lru_gen_refault(struct folio
tier = lru_tier_from_refs(refs);
atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]);
- mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta);
+ mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta);
/*
* Count the following two cases as stalls:
_
Patches currently in -mm which might be from kaleshsingh@xxxxxxxxxx are
mm-unstable-multi-gen-lru-fix-workingset-accounting.patch
