Patch "RDMA/cm: add timeout to cm_destroy_id wait" has been added to the 6.6-stable tree

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This is a note to let you know that I've just added the patch titled

    RDMA/cm: add timeout to cm_destroy_id wait

to the 6.6-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rdma-cm-add-timeout-to-cm_destroy_id-wait.patch
and it can be found in the queue-6.6 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 7708d543bee9078b6c92ebc67cb56882c52bbc21
Author: Manjunath Patil <manjunath.b.patil@xxxxxxxxxx>
Date:   Fri Mar 8 22:33:23 2024 -0800

    RDMA/cm: add timeout to cm_destroy_id wait
    
    [ Upstream commit 96d9cbe2f2ff7abde021bac75eafaceabe9a51fa ]
    
    Add timeout to cm_destroy_id, so that userspace can trigger any data
    collection that would help in analyzing the cause of delay in destroying
    the cm_id.
    
    New noinline function helps dtrace/ebpf programs to hook on to it.
    Existing functionality isn't changed except triggering a probe-able new
    function at every timeout interval.
    
    We have seen cases where CM messages stuck with MAD layer (either due to
    software bug or faulty HCA), leading to cm_id getting stuck in the
    following call stack. This patch helps in resolving such issues faster.
    
    kernel: ... INFO: task XXXX:56778 blocked for more than 120 seconds.
    ...
            Call Trace:
            __schedule+0x2bc/0x895
            schedule+0x36/0x7c
            schedule_timeout+0x1f6/0x31f
            ? __slab_free+0x19c/0x2ba
            wait_for_completion+0x12b/0x18a
            ? wake_up_q+0x80/0x73
            cm_destroy_id+0x345/0x610 [ib_cm]
            ib_destroy_cm_id+0x10/0x20 [ib_cm]
            rdma_destroy_id+0xa8/0x300 [rdma_cm]
            ucma_destroy_id+0x13e/0x190 [rdma_ucm]
            ucma_write+0xe0/0x160 [rdma_ucm]
            __vfs_write+0x3a/0x16d
            vfs_write+0xb2/0x1a1
            ? syscall_trace_enter+0x1ce/0x2b8
            SyS_write+0x5c/0xd3
            do_syscall_64+0x79/0x1b9
            entry_SYSCALL_64_after_hwframe+0x16d/0x0
    
    Signed-off-by: Manjunath Patil <manjunath.b.patil@xxxxxxxxxx>
    Link: https://lore.kernel.org/r/20240309063323.458102-1-manjunath.b.patil@xxxxxxxxxx
    Signed-off-by: Leon Romanovsky <leon@xxxxxxxxxx>
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index ff58058aeadca..bf0df6ee4f785 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -34,6 +34,7 @@ MODULE_AUTHOR("Sean Hefty");
 MODULE_DESCRIPTION("InfiniBand CM");
 MODULE_LICENSE("Dual BSD/GPL");
 
+#define CM_DESTROY_ID_WAIT_TIMEOUT 10000 /* msecs */
 static const char * const ibcm_rej_reason_strs[] = {
 	[IB_CM_REJ_NO_QP]			= "no QP",
 	[IB_CM_REJ_NO_EEC]			= "no EEC",
@@ -1025,10 +1026,20 @@ static void cm_reset_to_idle(struct cm_id_private *cm_id_priv)
 	}
 }
 
+static noinline void cm_destroy_id_wait_timeout(struct ib_cm_id *cm_id)
+{
+	struct cm_id_private *cm_id_priv;
+
+	cm_id_priv = container_of(cm_id, struct cm_id_private, id);
+	pr_err("%s: cm_id=%p timed out. state=%d refcnt=%d\n", __func__,
+	       cm_id, cm_id->state, refcount_read(&cm_id_priv->refcount));
+}
+
 static void cm_destroy_id(struct ib_cm_id *cm_id, int err)
 {
 	struct cm_id_private *cm_id_priv;
 	struct cm_work *work;
+	int ret;
 
 	cm_id_priv = container_of(cm_id, struct cm_id_private, id);
 	spin_lock_irq(&cm_id_priv->lock);
@@ -1135,7 +1146,14 @@ static void cm_destroy_id(struct ib_cm_id *cm_id, int err)
 
 	xa_erase(&cm.local_id_table, cm_local_id(cm_id->local_id));
 	cm_deref_id(cm_id_priv);
-	wait_for_completion(&cm_id_priv->comp);
+	do {
+		ret = wait_for_completion_timeout(&cm_id_priv->comp,
+						  msecs_to_jiffies(
+						  CM_DESTROY_ID_WAIT_TIMEOUT));
+		if (!ret) /* timeout happened */
+			cm_destroy_id_wait_timeout(cm_id);
+	} while (!ret);
+
 	while ((work = cm_dequeue_work(cm_id_priv)) != NULL)
 		cm_free_work(work);
 




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux