2023-09-19 7:38 GMT+09:00, Tom Talpey <tom@xxxxxxxxxx>: > Namjae, after building a 6.6.0-rc2 kernel to test here at the IOLab, > I was surprised to see the smbdirect connection break during the > Connectathon "special" tests. The basic tests all work fine, but shortly > after the special tests begin, I start seeing this on the server (this > is with softRoCE, though I see similar failures over softiWarp): I'll try to reproduce it tonight. I found no problems in testing with the Windows client last week. I will have to check with cifs.ko & softROCE. Thanks for your report! > > [ 1266.623187] rxe0: qp#17 do_complete: non-flush error status = 2 > [ 1266.623233] ksmbd: smb_direct: Recv error. status='local QP operation > error (2)' opcode=0 > [ 1266.623605] ksmbd: smb_direct: disconnected > [ 1266.623610] ksmbd: sock_read failed: -107 > [ 1266.628656] rxe0: qp#18 do_complete: non-flush error status = 2 > [ 1266.628684] ksmbd: smb_direct: Recv error. status='local QP operation > error (2)' opcode=0 > [ 1266.628820] ksmbd: smb_direct: disconnected > [ 1266.628824] ksmbd: sock_read failed: -107 > [ 1266.633354] rxe0: qp#19 do_complete: non-flush error status = 2 > [ 1266.633380] ksmbd: smb_direct: Recv error. status='local QP operation > error (2)' opcode=0 > [ 1266.633583] ksmbd: smb_direct: disconnected > > The local QP error 2 is IB_WC_LOC_QP_OP_ERR, which is a buffer error > of some sort, could be a receive buffer unavailable or maybe a length > overrun. Both of these seem highly improbable, because the "basic" tests > run fine. The client sees only a disconnection with IB_WC_REM_OP_ERR, > which is expected in this case. > > OTOH it could be a client send issue, maybe a too-large datagram or an > smbdirect credit overrun. But it's the server detecting the error, so > I'm starting there for now. > > This worked as recently as 6.5, definitely it was all fine in 6.4. I am > not yet able to drill down to the level of figuring out what SMB3 > payload was being received by ksmbd. > > Steve tells me you test over RDMA semi-often. Have you seen this? > Any ideas are welcome. > > Tom. >
