On Tue, 2019-03-05 at 13:09 +0800, Jiufei Xue wrote: > But now I have found something new that may helpful. When the problem > happened, rpc_status was set to -121 which returned from > nfs4_xdr_dec_open->decode_putfh. I think it is the cause that client > can > not update the stateid. > > kworker/u4:0-7 [001] .... 559.920542: nfs4_xdr_dec_open: > status=0 openstateid=40289:0xd4bdfeb7 > kworker/1:0-2929 [001] .... 559.920550: > nfs4_open_stateid_update: error=0 (OK) fileid=00:2b:12 > fhandle=0x1e4566cb stateid=40289:0xd4bdfeb7 > kworker/1:0-2929 [001] .... 559.920551: nfs4_open_release: > cancel=1 rpc_status=0 rpc_done=1 openstateid=40289:0xd4bdfeb7 > > nfsd-2741 [001] .... 559.920571: nfsd_compound_status: > op=1/6 OP_SEQUENCE status=0 > nfsd-2741 [001] .... 559.920575: nfsd_compound_status: > op=2/6 OP_PUTFH status=0 > nfsd-2741 [001] .... 559.920581: > nfsd_inc_and_copy_stateid: client 5c7df867:90134041 stateid > 00031b7d:00009d62 // 40290 > nfsd-2741 [001] .... 559.920584: nfsd_compound_status: > op=3/6 OP_OPEN status=0 > nfsd-2741 [001] .... 559.920584: nfsd_compound_status: > op=4/6 OP_GETFH status=0 > nfsd-2741 [001] .... 559.920585: nfsd_compound_status: > op=5/6 OP_ACCESS status=0 > nfsd-2741 [001] .... 559.920588: nfsd_compound_status: > op=6/6 OP_GETATTR status=0 > kworker/u4:0-7 [001] .... 559.920622: nfs4_xdr_dec_open: > status=-121 openstateid=0:0x7bd5c66f // added for debug > kworker/1:0-2929 [001] .... 559.920628: nfs4_open_release: > cancel=1 rpc_status=-121 rpc_done=1 openstateid=0:0x7bd5c66f // added > for debug > > And we got this error in dmesg: > [ 559.834808] nfs: Server returned operation 53 but we issued a > request for 22 > [ 559.835741] nfs: Server returned operation 53 but we issued a > request for 22 > > However, I don't kown why the client received two OP_SEQUENCE > response. > Any ideas? > No, that's quite weird... Were you able to catch this behaviour on wireshark? It would be interesting to see if the server is actually sending 2 SEQUENCE ops or if that is a corruption that occurs on the client. -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx