> -----Original Message----- > From: Shifeng Li <lishifeng1992@xxxxxxx> > Sent: Monday, November 20, 2023 2:31 AM > To: Ismail, Mustafa <mustafa.ismail@xxxxxxxxx> > Cc: Saleem, Shiraz <shiraz.saleem@xxxxxxxxx>; jgg@xxxxxxxx; > leon@xxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; > Ding, Hui <dinghui@xxxxxxxxxxxxxx>; Shifeng Li <lishifeng1992@xxxxxxx> > Subject: [PATCH] RDMA/irdma: Avoid free the non-cqp_request scratch > > When creating ceq_0 during probing irdma, cqp.sc_cqp will be sent as a > cqp_request to cqp->sc_cqp.sq_ring. If the request is pending when removing > the irdma driver or unplugging its aux device, cqp.sc_cqp will be dereferenced > as wrong struct in irdma_free_pending_cqp_request(). > > crash> bt 3669 > PID: 3669 TASK: ffff88aef892c000 CPU: 28 COMMAND: "kworker/28:0" > #0 [fffffe0000549e38] crash_nmi_callback at ffffffff810e3a34 > #1 [fffffe0000549e40] nmi_handle at ffffffff810788b2 > #2 [fffffe0000549ea0] default_do_nmi at ffffffff8107938f > #3 [fffffe0000549eb8] do_nmi at ffffffff81079582 > #4 [fffffe0000549ef0] end_repeat_nmi at ffffffff82e016b4 > [exception RIP: native_queued_spin_lock_slowpath+1291] > RIP: ffffffff8127e72b RSP: ffff88aa841ef778 RFLAGS: 00000046 > RAX: 0000000000000000 RBX: ffff88b01f849700 RCX: ffffffff8127e47e > RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffffffff83857ec0 > RBP: ffff88afe3e4efc8 R8: ffffed15fc7c9dfa R9: ffffed15fc7c9dfa > R10: 0000000000000001 R11: ffffed15fc7c9df9 R12: 0000000000740000 > R13: ffff88b01f849708 R14: 0000000000000003 R15: ffffed1603f092e1 > ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0000 > --- <NMI exception stack> --- > #5 [ffff88aa841ef778] native_queued_spin_lock_slowpath at ffffffff8127e72b > #6 [ffff88aa841ef7b0] _raw_spin_lock_irqsave at ffffffff82c22aa4 > #7 [ffff88aa841ef7c8] __wake_up_common_lock at ffffffff81257363 > #8 [ffff88aa841ef888] irdma_free_pending_cqp_request at ffffffffa0ba12cc > [irdma] > #9 [ffff88aa841ef958] irdma_cleanup_pending_cqp_op at ffffffffa0ba1469 > [irdma] > #10 [ffff88aa841ef9c0] irdma_ctrl_deinit_hw at ffffffffa0b2989f [irdma] > #11 [ffff88aa841efa28] irdma_remove at ffffffffa0b252df [irdma] > #12 [ffff88aa841efae8] auxiliary_bus_remove at ffffffff8219afdb > #13 [ffff88aa841efb00] device_release_driver_internal at ffffffff821882e6 > #14 [ffff88aa841efb38] bus_remove_device at ffffffff82184278 > #15 [ffff88aa841efb88] device_del at ffffffff82179d23 > #16 [ffff88aa841efc48] ice_unplug_aux_dev at ffffffffa0eb1c14 [ice] > #17 [ffff88aa841efc68] ice_service_task at ffffffffa0d88201 [ice] > #18 [ffff88aa841efde8] process_one_work at ffffffff811c589a > #19 [ffff88aa841efe60] worker_thread at ffffffff811c71ff > #20 [ffff88aa841eff10] kthread at ffffffff811d87a0 > #21 [ffff88aa841eff50] ret_from_fork at ffffffff82e0022f > > Fixes: 44d9e52977a1 ("RDMA/irdma: Implement device initialization > definitions") > Signed-off-by: Shifeng Li <lishifeng1992@xxxxxxx> > --- > drivers/infiniband/hw/irdma/utils.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/infiniband/hw/irdma/utils.c > b/drivers/infiniband/hw/irdma/utils.c > index 445e69e86409..222ec1f761a1 100644 > --- a/drivers/infiniband/hw/irdma/utils.c > +++ b/drivers/infiniband/hw/irdma/utils.c > @@ -541,7 +541,7 @@ void irdma_cleanup_pending_cqp_op(struct > irdma_pci_f *rf) > for (i = 0; i < pending_work; i++) { > cqp_request = (struct irdma_cqp_request *)(unsigned long) > cqp->scratch_array[wqe_idx]; > - if (cqp_request) > + if (cqp_request && cqp_request != (struct irdma_cqp_request > +*)&cqp->sc_cqp) > irdma_free_pending_cqp_request(cqp, cqp_request); > wqe_idx = (wqe_idx + 1) % IRDMA_RING_SIZE(cqp- > >sc_cqp.sq_ring); > } > -- > 2.25.1 Hi Li, Could you describe how you hit this issue? It seems like the probe would not have completed successfully if the create cceq request was still pending. A better fix might be to set the scratch to 0 in this case: In irdma_create_ceq(), pass 0 for scratch in call to irdma_sc_cceq_create(&iwceq->sc_ceq, 0), since it doesn't appear to be used in this case. Is it possible to test this fix? Thanks!