-----"Krishnamraju Eraparaju" <krishna2@xxxxxxxxxxx> wrote: ----- >To: "Bernard Metzler" <BMT@xxxxxxxxxxxxxx>, "Steve Wise" ><larrystevenwise@xxxxxxxxx> >From: "Krishnamraju Eraparaju" <krishna2@xxxxxxxxxxx> >Date: 09/17/2019 02:48PM >Cc: "Jason Gunthorpe" <jgg@xxxxxxxx>, "Sagi Grimberg" ><sagi@xxxxxxxxxxx>, "linux-rdma@xxxxxxxxxxxxxxx" ><linux-rdma@xxxxxxxxxxxxxxx>, "Nirranjan Kirubaharan" ><nirranjan@xxxxxxxxxxx> >Subject: [EXTERNAL] Re: Re: Re: [PATCH v3] iwcm: don't hold the irq >disabled lock on iw_rem_ref > >On Tuesday, September 09/17/19, 2019 at 14:34:24 +0530, Bernard >Metzler wrote: >> -----"Jason Gunthorpe" <jgg@xxxxxxxx> wrote: ----- >> >> >To: "Krishnamraju Eraparaju" <krishna2@xxxxxxxxxxx> >> >From: "Jason Gunthorpe" <jgg@xxxxxxxx> >> >Date: 09/16/2019 06:28PM >> >Cc: "Steve Wise" <larrystevenwise@xxxxxxxxx>, "Bernard Metzler" >> ><BMT@xxxxxxxxxxxxxx>, "Sagi Grimberg" <sagi@xxxxxxxxxxx>, >> >"linux-rdma@xxxxxxxxxxxxxxx" <linux-rdma@xxxxxxxxxxxxxxx> >> >Subject: [EXTERNAL] Re: Re: [PATCH v3] iwcm: don't hold the irq >> >disabled lock on iw_rem_ref >> > >> >On Wed, Sep 11, 2019 at 09:28:16PM +0530, Krishnamraju Eraparaju >> >wrote: >> >> Hi Steve & Bernard, >> >> >> >> Thanks for the review comments. >> >> I will do those formating changes. >> > >> >I don't see anything in patchworks, but the consensus is to drop >> >Sagi's patch pending this future patch? >> > >> >Jason >> > >> This is my impression as well. But consensus should be >> explicit...Sagi, what do you think? >> >> Best regards, >> Bernard. >> >While testing iSER(with my proposed patch applied) I see Chelsio >iwarp >driver is hitting the below deadlock issue. This is due to iw_rem_ref >reordering changes in IWCM. > >Bernard, how about replacing vmalloc/vfree with kmalloc/kfree, >such that freeing of SIW qp resources can be done with spinlocks >held? >to fix the orginal vfree issue less invasively.. Well, I'd really like to avoid kmalloc on potentially large data structures when there is no need to have physically contiguous memory. I could of course move that vfree out to a worker. Simple, but not really nice though. So it seems it would be no good option to restructure the Chelsio driver? Thanks Bernard. > >Steve, any suggestions? > > >[ 1230.161871] INFO: task kworker/u12:0:11291 blocked for more than >122 >seconds. >[ 1230.162147] Not tainted 5.3.0-rc5+ #19 >[ 1230.162417] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" >disables this message. >[ 1230.162911] kworker/u12:0 D13000 11291 2 0x80004080 >[ 1230.163186] Workqueue: iw_cm_wq cm_work_handler >[ 1230.163456] Call Trace: >[ 1230.163718] ? __schedule+0x297/0x510 >[ 1230.163986] schedule+0x2e/0x90 >[ 1230.164253] schedule_timeout+0x1c0/0x280 >[ 1230.164520] ? xas_store+0x23e/0x500 >[ 1230.164789] wait_for_completion+0xa2/0x110 >[ 1230.165067] ? wake_up_q+0x70/0x70 >[ 1230.165336] c4iw_destroy_qp+0x141/0x260 [iw_cxgb4] >[ 1230.165611] ? xas_store+0x23e/0x500 >[ 1230.165893] ? _cond_resched+0x10/0x20 >[ 1230.166160] ? wait_for_completion+0x2e/0x110 >[ 1230.166432] ib_destroy_qp_user+0x142/0x230 >[ 1230.166699] rdma_destroy_qp+0x1f/0x40 >[ 1230.166966] iser_free_ib_conn_res+0x52/0x190 [ib_iser] >[ 1230.167241] iser_cleanup_handler.isra.15+0x32/0x60 [ib_iser] >[ 1230.167510] iser_cma_handler+0x23b/0x730 [ib_iser] >[ 1230.167776] cma_iw_handler+0x154/0x1e0 >[ 1230.168037] cm_work_handler+0xb4c/0xd60 >[ 1230.168302] process_one_work+0x155/0x380 >[ 1230.168564] worker_thread+0x41/0x3b0 >[ 1230.168827] kthread+0xf3/0x130 >[ 1230.169086] ? process_one_work+0x380/0x380 >[ 1230.169350] ? kthread_bind+0x10/0x10 >[ 1230.169615] ret_from_fork+0x35/0x40 >[ 1230.169885] NMI backtrace for cpu 3 > >
