On 3/4/25 9:43 PM, Li Nan wrote: > We found a following problem in kernel 5.10, and the same problem should > exist in mainline: > > During NFS mount using 'soft' option over RoCE network, we observed kernel > crash with below trace when network issues occur (congestion/disconnect): > nfs: server 10.10.253.211 not responding, timed out > BUG: kernel NULL pointer dereference, address: 00000000000000a0 > RIP: 0010:frwr_unmap_sync+0x77/0x200 [rpcrdma] > Call Trace: > ? __die_body.cold+0x8/0xd > ? no_context+0x155/0x230 > ? __bad_area_nosemaphore+0x52/0x1a0 > ? exc_page_fault+0x2dc/0x550 > ? asm_exc_page_fault+0x1e/0x30 > ? frwr_unmap_sync+0x77/0x200 [rpcrdma] > xprt_release+0x9e/0x1a0 [sunrpc] > rpc_release_resources_task+0xe/0x50 [sunrpc] > rpc_release_task+0x19/0xa0 [sunrpc] > rpc_async_schedule+0x29/0x40 [sunrpc] > process_one_work+0x1b2/0x350 > worker_thread+0x49/0x310 > ? rescuer_thread+0x380/0x380 > kthread+0xfb/0x140 > > Problem analysis: > The crash happens in frwr_unmap_sync() when accessing req->rl_registered > list, caused by either NULL pointer or accessing freed MR resources. > There's a race condition between: > T1 > __ib_process_cq > wc->wr_cqe->done (frwr_wc_localinv) > rpcrdma_flush_disconnect > rpcrdma_force_disconnect > xprt_force_disconnect > xprt_autoclose > xprt_rdma_close > rpcrdma_xprt_disconnect > rpcrdma_reqs_reset > frwr_reset > rpcrdma_mr_pop(&req->rl_registered) > T2 > rpc_async_schedule > rpc_release_task > rpc_release_resources_task > xprt_release > xprt_rdma_free > frwr_unmap_sync > rpcrdma_mr_pop(&req->rl_registered) > > This problem also exists in function rpcrdma_mrs_destroy(). > Dai, is this the same as the system test problem you've been looking at? -- Chuck Lever
