Re: [PATCH] soundwire: cadence: fix race condition between suspend and Slave device alerts

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 8/19/20 4:06 AM, Vinod Koul wrote:
On 18-08-20, 06:23, Bard Liao wrote:
From: Pierre-Louis Bossart <pierre-louis.bossart@xxxxxxxxxxxxxxx>

In system suspend stress cases, the SOF CI reports timeouts. The root
cause is that an alert is generated while the system suspends. The
interrupt handling generates transactions on the bus that will never
be handled because the interrupts are disabled in parallel.

As a result, the transaction never completes and times out on resume.
This error doesn't seem too problematic since it happens in a work
queue, and the system recovers without issues.

Nevertheless, this race condition should not happen. When doing a
system suspend, or when disabling interrupts, we should make sure the
current transaction can complete, and prevent new work from being
queued.

BugLink: https://github.com/thesofproject/linux/issues/2344
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@xxxxxxxxxxxxxxx>
Reviewed-by: Ranjani Sridharan <ranjani.sridharan@xxxxxxxxxxxxxxx>
Reviewed-by: Rander Wang <rander.wang@xxxxxxxxxxxxxxx>
Signed-off-by: Bard Liao <yung-chuan.liao@xxxxxxxxxxxxxxx>
---
  drivers/soundwire/cadence_master.c | 24 +++++++++++++++++++++++-
  drivers/soundwire/cadence_master.h |  1 +
  2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/soundwire/cadence_master.c b/drivers/soundwire/cadence_master.c
index 24eafe0aa1c3..1330ffc47596 100644
--- a/drivers/soundwire/cadence_master.c
+++ b/drivers/soundwire/cadence_master.c
@@ -791,7 +791,16 @@ irqreturn_t sdw_cdns_irq(int irq, void *dev_id)
  			     CDNS_MCP_INT_SLAVE_MASK, 0);
int_status &= ~CDNS_MCP_INT_SLAVE_MASK;
-		schedule_work(&cdns->work);
+
+		/*
+		 * Deal with possible race condition between interrupt
+		 * handling and disabling interrupts on suspend.
+		 *
+		 * If the master is in the process of disabling
+		 * interrupts, don't schedule a workqueue
+		 */
+		if (cdns->interrupt_enabled)
+			schedule_work(&cdns->work);

would it not make sense to mask the interrupts first and then cancel the
work? that way you are guaranteed that after this call you dont have
interrupts and work scheduled?

cancel_work_sync() will either
a) wait until the current work completes, or
b) prevent a new one from starting.

there's no way to really 'abort' a workqueue, 'cancel' means either complete or don't start.

if you disable the interrupts then cancel the work, you have a risk of not letting the work complete if it already started (case a).

The race is
a) the interrupt thread (this function) starts
b) the work is scheduled and starts
c) the suspend handler starts and disables interrupts in [1] below.
d) the work initiates transactions which will never complete since Cadence interrupts have been disabled.

So the idea was that before disabling interrupts, the suspend handler changes the status, and then calls cancel_work_sync(). the status is also used to prevent a new work from being scheduled if you already know the suspend is on-going. The test on the status above is not strictly necessary, I believe the sequence is safe without it but it avoid starting a useless work.

If you want to follow the flow it's better to start with what the suspend handler does below first, then look at how the interrupt thread might interfere. The diff format does not help, might be also easier to apply the patch and look at the rest of the code, e.g the 3 mask updates mentioned below are not included in the diff.


  	}
cdns_writel(cdns, CDNS_MCP_INTSTAT, int_status);
@@ -924,6 +933,19 @@ int sdw_cdns_enable_interrupt(struct sdw_cdns *cdns, bool state)
  		slave_state = cdns_readl(cdns, CDNS_MCP_SLAVE_INTSTAT1);
  		cdns_writel(cdns, CDNS_MCP_SLAVE_INTSTAT1, slave_state);
  	}

[1]

+	cdns->interrupt_enabled = state;
+
+	/*
+	 * Complete any on-going status updates before updating masks,
+	 * and cancel queued status updates.
+	 *
+	 * There could be a race with a new interrupt thrown before
+	 * the 3 mask updates below are complete, so in the interrupt
+	 * we use the 'interrupt_enabled' status to prevent new work
+	 * from being queued.
+	 */
+	if (!state)
+		cancel_work_sync(&cdns->work);
cdns_writel(cdns, CDNS_MCP_SLAVE_INTMASK0, slave_intmask0);
  	cdns_writel(cdns, CDNS_MCP_SLAVE_INTMASK1, slave_intmask1);
diff --git a/drivers/soundwire/cadence_master.h b/drivers/soundwire/cadence_master.h
index fdec62b912d3..4d1aab5b5ec2 100644
--- a/drivers/soundwire/cadence_master.h
+++ b/drivers/soundwire/cadence_master.h
@@ -133,6 +133,7 @@ struct sdw_cdns {
bool link_up;
  	unsigned int msg_count;
+	bool interrupt_enabled;
struct work_struct work; --
2.17.1




[Index of Archives]     [ALSA User]     [Linux Audio Users]     [Pulse Audio]     [Kernel Archive]     [Asterisk PBX]     [Photo Sharing]     [Linux Sound]     [Video 4 Linux]     [Gimp]     [Yosemite News]

  Powered by Linux