Even though the parallel_data "pd" instance passed to padata_reorder() is
guaranteed to exist as per the reference held by its callers, the same is
not true for the associated padata_shell, pd->ps. More specifically, once
the last padata_priv request has been completed, either at entry from
padata_reorder() or concurrently to it, the padata API users are well
within their right to free the padata_shell instance.
Note that this is a purely theoretical issue, it has not been actually
observed. Yet it's worth fixing for the sake of robustness.
Exploit the fact that as long as there are any not yet completed
padata_priv's around on any of the percpu reorder queues, pd->ps is
guaranteed to exist. Make padata_reorder() to load from pd->ps only when
it's known that there is at least one in-flight padata_priv object to
reorder. Note that this involves moving pd->ps accesses to under the
reorder->lock as appropriate, so that the found padata_priv object won't
get dequeued and completed concurrently from a different context.
Fixes: bbefa1dd6a6d ("crypto: pcrypt - Avoid deadlock by using per-instance
padata queues")
Signed-off-by: Nicolai Stange <nstange@xxxxxxx>
---
kernel/padata.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/kernel/padata.c b/kernel/padata.c
index e9eab3e94cfc..fa4818b81eca 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -286,7 +286,6 @@ static struct padata_priv *padata_dequeue_next(struct parallel_data *pd)
static bool padata_reorder(struct parallel_data *pd)
{
- struct padata_instance *pinst = pd->ps->pinst;
int cb_cpu;
struct padata_priv *padata;
struct padata_serial_queue *squeue;
@@ -323,7 +322,11 @@ static bool padata_reorder(struct parallel_data *pd)
list_add_tail(&padata->list, &squeue->serial.list);
spin_unlock(&squeue->serial.lock);
- queue_work_on(cb_cpu, pinst->serial_wq, &squeue->work);
+ /*
+ * Note: as long as there are requests in-flight,
+ * pd->ps is guaranteed to exist.
+ */
+ queue_work_on(cb_cpu, pd->ps->pinst->serial_wq, &squeue->work);
}
spin_unlock_bh(&pd->lock);
@@ -340,14 +343,23 @@ static bool padata_reorder(struct parallel_data *pd)
reorder = per_cpu_ptr(pd->reorder_list, pd->cpu);
if (!list_empty(&reorder->list)) {
+ bool reenqueued;
+
spin_lock(&reorder->lock);
if (!__padata_find_next(pd, reorder)) {
spin_unlock(&reorder->lock);
return false;
}
+
+ /*
+ * Note: as long as there are requests in-flight,
+ * pd->ps is guaranteed to exist.
+ */
+ reenqueued = queue_work(pd->ps->pinst->serial_wq,
+ &pd->reorder_work);
spin_unlock(&reorder->lock);
- return queue_work(pinst->serial_wq, &pd->reorder_work);
+ return reenqueued;
}
return false;
--
2.37.3
