Hi Matthew, On 11/6/20 01:26, Matthew Wilcox (Oracle) wrote: > This attempts to address some of the criticisms of Tim's original > patch back in June: > > https://lore.kernel.org/linux-mm/d1cc9f12a8ad6c2a52cb600d93b06b064f2bbc57.1593205965.git.tim.c.chen@xxxxxxxxxxxxxxx/ > > I don't love how messy this is. Better suggestions (dynamic allocation of > pagevecs?) welcome. It'd be even better if we could measure contention > and resize the LRUvecs on demand. > > I do intend to add pagevec_alloc() and pagevec_free(), but that's to > solve a different problem. Sorry for bring this old thread back. We hit the lru lock contention when we run redis related test. And both Tim's patch and this patchset (with a little bit update to cover fbatch) can reduce the lru lock contention and improve the redis test result: commit: v6.4-rc4 f1989218504dfe1ab4bdad3b8c0422f57c7698db (v6.4-rc4 + this patchset) v6.4-rc4 f1989218504dfe1ab4bdad3b8c0 ---------------- --------------------------- 117.24 -2.1% 114.73 memtier.avg_gets_avg_latency_ms 2.78 +2.9% 2.86 memtier.avg_gets_hits/s 2847 +2.9% 2930 memtier.avg_gets_kb/s 2.78 +2.9% 2.86 memtier.avg_gets_ops/s 118.48 -2.5% 115.57 memtier.avg_gets_p50_latency_ms 139.58 -4.1% 133.90 memtier.avg_gets_p99.9_latency_ms 139.58 -4.1% 133.90 memtier.avg_gets_p99_latency_ms 84.18 -3.0% 81.69 memtier.avg_sets_avg_latency_ms 287220 +2.9% 295580 memtier.avg_sets_kb/s 280.48 +2.9% 288.64 memtier.avg_sets_ops/s 83.48 -2.3% 81.57 memtier.avg_sets_p50_latency_ms 150.50 -8.7% 137.40 memtier.avg_sets_p99.9_latency_ms 124.46 -7.4% 115.30 memtier.avg_sets_p99_latency_ms 84.51 -2.9% 82.01 memtier.avg_totals_avg_latency_ms 2.78 +2.9% 2.86 memtier.avg_totals_hits/s 290068 +2.9% 298511 memtier.avg_totals_kb/s 283.26 +2.9% 291.50 memtier.avg_totals_ops/s 83.51 -2.3% 81.60 memtier.avg_totals_p50_latency_ms 152.48 -7.6% 140.93 memtier.avg_totals_p99.9_latency_ms 127.32 -5.5% 120.30 memtier.avg_totals_p99_latency_ms 127.48 -6.4% 119.29 memtier.get_latency_90%_ms 128.51 -7.2% 119.29 memtier.get_latency_95%_ms 128.51 -7.2% 119.29 memtier.get_latency_99%_ms 128.51 -7.2% 119.29 memtier.get_latency_99.9%_ms 96.76 -6.9% 90.11 memtier.set_latency_90%_ms 105.47 -9.7% 95.23 memtier.set_latency_95%_ms 124.41 -6.6% 116.22 memtier.set_latency_99%_ms 162.81 -1.9% 159.74 memtier.set_latency_99.9%_ms 8.08 +0.6% 8.13 memtier.time.elapsed_time 8.08 +0.6% 8.13 memtier.time.elapsed_time.max 79.00 -11.4% 70.00 memtier.time.involuntary_context_switches 40744 +1.5% 41368 memtier.time.maximum_resident_set_size 61696 +17.2% 72289 memtier.time.minor_page_faults 4096 +0.0% 4096 memtier.time.page_size 71.00 -5.6% 67.00 memtier.time.percent_of_cpu_this_job_got 4.01 -5.0% 3.81 memtier.time.system_time 1.77 -6.8% 1.65 memtier.time.user_time 70.00 +164.3% 185.00 memtier.time.voluntary_context_switches 13130 -2.1% 12850 memtier.total_gets_avg_latency_ms 311.43 +2.9% 320.55 memtier.total_gets_hits/s 318951 +2.9% 328234 memtier.total_gets_kb/s 311.43 +2.9% 320.55 memtier.total_gets_ops/s 13269 -2.5% 12944 memtier.total_gets_p50_latency_ms 15632 -4.1% 14996 memtier.total_gets_p99.9_latency_ms 15632 -4.1% 14996 memtier.total_gets_p99_latency_ms 9428 -3.0% 9149 memtier.total_sets_avg_latency_ms 32168738 +2.9% 33105003 memtier.total_sets_kb/s 31413 +2.9% 32327 memtier.total_sets_ops/s 9350 -2.3% 9135 memtier.total_sets_p50_latency_ms 16855 -8.7% 15388 memtier.total_sets_p99.9_latency_ms 13939 -7.4% 12913 memtier.total_sets_p99_latency_ms 9464 -2.9% 9185 memtier.total_totals_avg_latency_ms 311.43 +2.9% 320.55 memtier.total_totals_hits/s 32487689 +2.9% 33433237 memtier.total_totals_kb/s 31724 +2.9% 32648 memtier.total_totals_ops/s 9353 -2.3% 9139 memtier.total_totals_p50_latency_ms 17078 -7.6% 15784 memtier.total_totals_p99.9_latency_ms 14259 -5.5% 13473 memtier.total_totals_p99_latency_ms We can see both throughput and latency are benefited. And the lru lock contention is reduced from 13.11% to 1.79%: 13.11 -11.3 1.79 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add _lru_vma.do_anonymous_page We noticed there are updated version here: https://lore.kernel.org/linux-mm/20201106123040.28451-1-willy@xxxxxxxxxxxxx/ But didn't hit upstream kernel. I am wondering what exactly block this patchset merged. Thanks. Regards Yin, Fengwei > > Matthew Wilcox (Oracle) (2): > pagevec: Allow pagevecs to be different sizes > pagevec: Increase the size of LRU pagevecs > > include/linux/pagevec.h | 25 +++++++++++++++++++++---- > mm/swap.c | 23 +++++++++++++++++------ > 2 files changed, 38 insertions(+), 10 deletions(-) >
