To Huang Ying,
I tried to apply migrc to swap. Fortunately I couldn't see any
regression but no performance improvement either. I thought it's
meaningless to make code bigger without observing any benefit. So I
won't include the part. Thoughts?
To Nadav Amit,
I tried to split this patch set to as many as possible for better
review. However, it was very hard to make each patch meaningfully and
stably work because it works very tightly coupled to one another. So I
ended in combining those patches to one again. Instead, I tried my best
to add sufficient comments in code. Any opinion would be appreciated.
Hi everyone,
While I'm working with CXL memory, I have been facing migraion overhead
esp. TLB shootdown on promotion or demotion between different tiers.
Yeah.. most TLB shootdowns on migration through hinting fault can be
avoided thanks to Huang Ying's work, commit 4d4b6d66db ("mm,unmap: avoid
flushing TLB in batch if PTE is inaccessible").
However, it's only for ones using hinting fault. I thought it'd be much
better if we have a general mechanism to reduce # of TLB flushes and
TLB misses, that we can apply to any type of migration. I tried it only
for tiering migration for now tho.
I'm suggesting a mechanism to reduce TLB flushes by keeping source and
destination of folios participated in the migrations until all TLB
flushes required are done, only if those folios are not mapped with
write permission PTE entries at all. I worked Based on v6.6-rc5.
Can you believe it? I saw the number of TLB full flush reduced about
80%, iTLB miss reduced about 50% and the performance improved a little
bit with the workload I tested with, XSBench. However, I believe that it
would help more with other ones or any real ones. It'd be appreciated to
let me know if I'm missing something.
Byungchul
---
Changes from RFC v2:
1. Remove additional occupation in struct page. To do that,
unioned with lru field for migrc's list and added a page
flag. I know page flag is a thing that we don't like to add
but no choice because migrc should distinguish folios under
migrc's control from others. Instead, I force migrc to be
used only on 64 bit system to mitigate you guys from getting
angry.
2. Remove meaningless internal object allocator that I
introduced to minimize impact onto the system. However, a ton
of tests showed there was no difference.
3. Stop migrc from working when the system is in high memory
pressure like about to perform direct reclaim. At the
condition where the swap mechanism is heavily used, I found
the system suffered from regression without this control.
4. Exclude folios that pte_dirty() == true from migrc's interest
so that migrc can work simpler.
5. Combine several patches that work tightly coupled to one.
6. Add sufficient comments for better review.
7. Manage migrc's request in per-node manner (from globally).
8. Add TLB miss improvement in commit message.
9. Test with more CPUs(4 -> 16) to see bigger improvement.
Changes from RFC:
1. Fix a bug triggered when a destination folio at the previous
migration becomes a source folio at the next migration,
before the folio gets handled properly so that the folio can
play with another migration. There was inconsistency in the
folio's state. Fixed it.
2. Split the patch set into more pieces so that the folks can
review better. (Feedbacked by Nadav Amit)
3. Fix a wrong usage of barrier e.g. smp_mb__after_atomic().
(Feedbacked by Nadav Amit)
4. Tried to add sufficient comments to explain the patch set
better. (Feedbacked by Nadav Amit)
Byungchul Park (3):
mm/rmap: Recognize non-writable TLB entries during TLB batch flush
mm: Defer TLB flush by keeping both src and dst folios at migration
mm, migrc: Add a sysctl knob to enable/disable MIGRC mechanism
arch/x86/include/asm/tlbflush.h | 9 +
arch/x86/mm/tlb.c | 98 ++++++++++
include/linux/mm.h | 25 +++
include/linux/mm_types.h | 49 +++++
include/linux/mm_types_task.h | 4 +-
include/linux/mmzone.h | 7 +
include/linux/page-flags.h | 7 +
include/linux/sched.h | 9 +
include/trace/events/mmflags.h | 9 +-
init/Kconfig | 14 ++
mm/internal.h | 14 ++
mm/memory.c | 13 ++
mm/migrate.c | 310 ++++++++++++++++++++++++++++++++
mm/page_alloc.c | 29 ++-
mm/rmap.c | 115 +++++++++++-
15 files changed, 703 insertions(+), 9 deletions(-)
--
2.17.1
