On Fri, 2020-11-13 at 22:30 +0100, Mkrtchyan, Tigran wrote: > > After more testing, it looks like that client doesn't like > notification bitmap: > > > [31576.789492] --> _nfs4_proc_getdeviceinfo > [31576.789503] --> nfs41_call_sync_prepare data->seq_server > 000000001d17c43e > [31576.789507] --> nfs4_alloc_slot used_slots=0000 > highest_used=4294967295 max_slots=16 > [31576.789510] <-- nfs4_alloc_slot used_slots=0001 highest_used=0 > slotid=0 > [31576.789527] encode_sequence: > sessionid=2910695007:150995712:0:16777216 seqid=92462 slotid=0 > max_slotid=0 cache_this=0 > [31576.789991] decode_getdeviceinfo: unsupported notification According to this, you appear to be returning a deviceinfo bitmap with at least one non-zero entry that is not in the first 32-bit word. We only ask for notifications for NOTIFY_DEVICEID4_CHANGE and NOTIFY_DEVICEID4_DELETE, so we only expect bitmap[0] to have non-zero entries. > [31576.790003] --> nfs4_alloc_slot used_slots=0001 highest_used=0 > max_slots=16 > [31576.790007] <-- nfs4_alloc_slot used_slots=0003 highest_used=1 > slotid=1 > [31576.790010] nfs4_free_slot: slotid 1 highest_used_slotid 0 > [31576.790013] nfs41_sequence_process: Error 0 free the slot > [31576.790017] nfs4_free_slot: slotid 0 highest_used_slotid > 4294967295 > [31576.790030] <-- _nfs4_proc_getdeviceinfo status=-5 > [31576.790034] nfs4_get_device_info getdevice info returns -5 > [31576.790084] <-- nfs4_get_device_info d 0000000000000000 > > > Tigran. > > > ----- Original Message ----- > > From: "Tigran Mkrtchyan" <tigran.mkrtchyan@xxxxxxx> > > To: "trondmy" <trondmy@xxxxxxxxxxxxxxx> > > Cc: "linux-nfs" <linux-nfs@xxxxxxxxxxxxxxx> > > Sent: Friday, 13 November, 2020 13:48:07 > > Subject: Re: [PATCH v3 00/11] Add RDMA support to the pNFS > > file+flexfiles data channels > > > Hi Trond, > > > > whit this changes (3ee69a14f92d74ced2647140b3799511ba4f3fa5) I see > > an infinite > > loop > > of LAYOUTGET->GETDEVICEINFO->LAYOUTRETURN without any attempt to > > connect to a > > DS. > > > > This is how the response to LAYOUTGET looks like. > > > > Network File System, Ops(2): SEQUENCE GETDEVINFO > > [Program Version: 4] > > [V4 Procedure: COMPOUND (1)] > > Status: NFS4_OK (0) > > Tag: <EMPTY> > > length: 0 > > contents: <EMPTY> > > Operations (count: 2) > > Opcode: SEQUENCE (53) > > Opcode: GETDEVINFO (47) > > Status: NFS4_OK (0) > > layout type: LAYOUT4_FLEX_FILES (4) > > r_netid: tcp > > length: 3 > > contents: tcp > > fill bytes: opaque data > > r_addr: 131.169.191.143.125.49 > > length: 22 > > contents: 131.169.191.143.125.49 > > fill bytes: opaque data > > version: 4 > > minorversion: 1 > > max_rsize: 1048576 > > max_wsize: 1048576 > > tightly coupled: Yes > > notify_mask: 0x00000006 (Change, Delete) > > notify_type: Change (1) > > notify_type: Delete (2) > > [Main Opcode: GETDEVINFO (47)] > > > > > > > > The MDS is mounted with IPv4. I can provide the full packet trace, > > if needed. > > > > > > Regards, > > Tigran. > > > > > > ----- Original Message ----- > > > From: "trondmy" <trondmy@xxxxxxxxxxxxxxx> > > > To: "linux-nfs" <linux-nfs@xxxxxxxxxxxxxxx> > > > Sent: Wednesday, 11 November, 2020 00:42:31 > > > Subject: Re: [PATCH v3 00/11] Add RDMA support to the pNFS > > > file+flexfiles data > > > channels > > > > > On Tue, 2020-11-10 at 18:18 -0500, trondmy@xxxxxxxxxx wrote: > > > > From: Trond Myklebust <trond.myklebust@xxxxxxxxxxxxxxx> > > > > > > > > Add support for connecting to the pNFS files/flexfiles data > > > > servers > > > > through RDMA, assuming that the GETDEVICEINFO call advertises > > > > that > > > > support. > > > > > > > > v2: Fix layoutstats encoding for pNFS/flexfiles. > > > > v3: Move most of the netid handling into the SUNRPC and RDMA > > > > modules. > > > > Fix up the mount code to benefit more from automated > > > > loading of > > > > SUNRPC transport modules. > > > > > > > > > > Note that one cleanup that I did not perform, but which really > > > could be > > > useful should we want to add more transport mechanisms, is to > > > move the > > > code to parse stringified addresses (in particular IETF style > > > "universal addresses") into the transport modules so that the > > > actual > > > parsing of mount and pNFS transport info can be automatically > > > extended > > > when new transport modules are added. > > > > > > -- > > > Trond Myklebust > > > Linux NFS client maintainer, Hammerspace > > > trond.myklebust@xxxxxxxxxxxxxxx -- Trond Myklebust CTO, Hammerspace Inc 4984 El Camino Real, Suite 208 Los Altos, CA 94022 www.hammer.space
