Re: [RFC PATCH] mm: swap: remove lru drain waiters

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 01/06/2020 17.37, Hillf Danton wrote:

After updating the lru drain sequence, new comers avoid waiting for
the current drainer, because he is flushing works on each online CPU,
by trying to lock the mutex; the drainer OTOH tries to do works for
those who fail to acquire the lock by checking the lru drain sequence
after releasing lock.

See eef1a429f234 ("mm/swap.c: piggyback lru_add_drain_all() calls")
for reasons why we can skip waiting for the lock.

That patch tells nothing about such change in behaviour.

Callers like invalidate_bdev() really need synchronous drain to be sure
that pages have no extra reference from per-cpu vectors.


The memory barriers around the sequence and the lock come together
to remove waiters without their drain works bandoned.

Cc: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
Cc: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxxxxxx>
Signed-off-by: Hillf Danton <hdanton@xxxxxxxx>
---
This is inspired by one of the works from Sebastian.

--- a/mm/swap.c
+++ b/mm/swap.c
@@ -714,10 +714,11 @@ static void lru_add_drain_per_cpu(struct
   */
  void lru_add_drain_all(void)
  {
-	static seqcount_t seqcount = SEQCNT_ZERO(seqcount);
+	static unsigned int lru_drain_seq;
  	static DEFINE_MUTEX(lock);
  	static struct cpumask has_work;
-	int cpu, seq;
+	int cpu;
+	unsigned int seq;
/*
  	 * Make sure nobody triggers this path before mm_percpu_wq is fully
@@ -726,18 +727,16 @@ void lru_add_drain_all(void)
  	if (WARN_ON(!mm_percpu_wq))
  		return;
- seq = raw_read_seqcount_latch(&seqcount);
+	lru_drain_seq++;
+	smp_mb();
- mutex_lock(&lock);
+more_work:
- /*
-	 * Piggyback on drain started and finished while we waited for lock:
-	 * all pages pended at the time of our enter were drained from vectors.
-	 */
-	if (__read_seqcount_retry(&seqcount, seq))
-		goto done;
+	if (!mutex_trylock(&lock))
+		return;
- raw_write_seqcount_latch(&seqcount);
+	smp_mb();
+	seq = lru_drain_seq;
cpumask_clear(&has_work); @@ -759,8 +758,11 @@ void lru_add_drain_all(void)
  	for_each_cpu(cpu, &has_work)
  		flush_work(&per_cpu(lru_add_drain_work, cpu));
-done:
  	mutex_unlock(&lock);
+
+	smp_mb();
+	if (seq != lru_drain_seq)
+		goto more_work;
  }
  #else
  void lru_add_drain_all(void)
--





[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux