Subject: + cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5.patch added to -mm tree
To: srivatsa.bhat@xxxxxxxxxxxxxxxxxx,fweisbec@xxxxxxxxx,oleg@xxxxxxxxxx,peterz@xxxxxxxxxxxxx,tj@xxxxxxxxxx
From: akpm@xxxxxxxxxxxxxxxxxxxx
Date: Tue, 20 May 2014 13:23:48 -0700
The patch titled
Subject: CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline
has been added to the -mm tree. Its filename is
cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5.patch
This patch should soon appear at
http://ozlabs.org/~akpm/mmots/broken-out/cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5.patch
and later at
http://ozlabs.org/~akpm/mmotm/broken-out/cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/SubmitChecklist when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Srivatsa S. Bhat <srivatsa.bhat@xxxxxxxxxxxxxxxxxx>
Subject: CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline
During CPU offline, in the stop-machine loop, we use 2 separate stages to
disable interrupts, to ensure that the CPU going offline doesn't get any
new IPIs from the other CPUs after it has gone offline.
However, an IPI sent much earlier might arrive late on the target CPU
(possibly _after_ the CPU has gone offline) due to hardware latencies, and
due to this, the smp-call-function callbacks queued on the outgoing CPU
might not get noticed (and hence not executed) at all.
This is somewhat theoretical, but in any case, it makes sense to
explicitly loop through the call_single_queue and flush any pending
callbacks before the CPU goes completely offline. So, flush the queued
smp-call-function callbacks in the MULTI_STOP_DISABLE_IRQ_ACTIVE stage,
after disabling interrupts on the active CPU. This can be trivially
achieved by invoking the generic_smp_call_function_single_interrupt()
function itself (and since the outgoing CPU is still online at this point,
we won't trigger the "IPI to offline CPU" warning in this function; so we
are safe to call it here).
This way, we would have handled all the queued callbacks before going
offline, and also, no new IPIs can be sent by the other CPUs to the
outgoing CPU at that point, because they will all be executing the
stop-machine code with interrupts disabled.
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@xxxxxxxxxxxxxxxxxx>
Suggested-by: Frederic Weisbecker <fweisbec@xxxxxxxxx>
Reviewed-by: Tejun Heo <tj@xxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Oleg Nesterov <oleg@xxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
include/linux/smp.h | 2 ++
kernel/smp.c | 17 ++++++++++++++---
kernel/stop_machine.c | 11 +++++++++++
3 files changed, 27 insertions(+), 3 deletions(-)
diff -puN include/linux/smp.h~cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5 include/linux/smp.h
--- a/include/linux/smp.h~cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5
+++ a/include/linux/smp.h
@@ -151,6 +151,8 @@ smp_call_function_any(const struct cpuma
static inline void kick_all_cpus_sync(void) { }
+static inline void generic_smp_call_function_single_interrupt(void) { }
+
#endif /* !SMP */
/*
diff -puN kernel/smp.c~cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5 kernel/smp.c
--- a/kernel/smp.c~cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5
+++ a/kernel/smp.c
@@ -177,9 +177,18 @@ static int generic_exec_single(int cpu,
return 0;
}
-/*
- * Invoked by arch to handle an IPI for call function single. Must be
- * called from the arch with interrupts disabled.
+/**
+ * generic_smp_call_function_single_interrupt - Execute SMP IPI callbacks
+ *
+ * Invoked by arch to handle an IPI for call function single.
+ *
+ * This is also invoked by a CPU about to go offline, to flush any pending
+ * smp-call-function callbacks queued on this CPU (including those for which
+ * the source CPU's IPIs might not have been received on this CPU yet).
+ * This ensures that all pending IPI callbacks are run before the CPU goes
+ * completely offline.
+ *
+ * Must be called with interrupts disabled.
*/
void generic_smp_call_function_single_interrupt(void)
{
@@ -187,6 +196,8 @@ void generic_smp_call_function_single_in
struct call_single_data *csd, *csd_next;
static bool warned;
+ WARN_ON(!irqs_disabled());
+
entry = llist_del_all(&__get_cpu_var(call_single_queue));
entry = llist_reverse_order(entry);
diff -puN kernel/stop_machine.c~cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5 kernel/stop_machine.c
--- a/kernel/stop_machine.c~cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5
+++ a/kernel/stop_machine.c
@@ -21,6 +21,7 @@
#include <linux/smpboot.h>
#include <linux/atomic.h>
#include <linux/lglock.h>
+#include <linux/smp.h>
/*
* Structure to determine completion condition and record errors. May
@@ -223,6 +224,16 @@ static int multi_cpu_stop(void *data)
if (is_active) {
local_irq_disable();
hard_irq_disable();
+
+ /*
+ * IPIs (from the inactive CPUs) might
+ * arrive late due to hardware latencies.
+ * So flush out any pending IPI callbacks
+ * explicitly, to ensure that the outgoing
+ * CPU doesn't go offline with work still
+ * pending (during CPU hotplug).
+ */
+ generic_smp_call_function_single_interrupt();
}
break;
case MULTI_STOP_RUN:
_
Patches currently in -mm which might be from srivatsa.bhat@xxxxxxxxxxxxxxxxxx are
smp-print-more-useful-debug-info-upon-receiving-ipi-on-an-offline-cpu.patch
smp-print-more-useful-debug-info-upon-receiving-ipi-on-an-offline-cpu-fix.patch
smp-print-more-useful-debug-info-upon-receiving-ipi-on-an-offline-cpu-v5.patch
cpu-hotplug-stop-machine-plug-race-window-that-leads-to-ipi-to-offline-cpu.patch
cpu-hotplug-stop-machine-plug-race-window-that-leads-to-ipi-to-offline-cpu-v3.patch
cpu-hotplug-stop-machine-plug-race-window-that-leads-to-ipi-to-offline-cpu-v5.patch
cpu-hotplug-smp-flush-any-pending-ipi-callbacks-before-cpu-offline-v5.patch
linux-next.patch
--
To unsubscribe from this list: send the line "unsubscribe mm-commits" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
