On Mon, Apr 08, 2013 at 01:57:55PM -0400, Michael Brown wrote:
> I'm trying to get Oracle's DNFS working against gluster's internal NFS
> and I've run into a snag. After Oracle mounts the exported NFS
> filesystem the FSINFO call fails.
>
> Let's look with wireshark:
...
> *Remote Procedure Call, Type:Call XID:0x47349478*
> Program: NFS (100003)
> Program Version: 3
> Procedure: FSINFO (19)
> Network File System, FSINFO Call DH:0x10650fe6
> [Program Version: 3]
> [V3 Procedure: FSINFO (19)]
> object
> length: 34
> [hash (CRC-32): 0x10650fe6]
> [Name: 192.168.10.1:/gv0/fleming3/db0/ALTUS_data]
> filehandle:
> 3a4f20117b487f884f169490a0349afacf71331965f573144e93b289a395148edfe5
Bytes of the object parameter:
< length > < 10 bytes >
00a0 .. .. 00 00 00 22 3a 4f 20 11 7b 48 7f 88 4f 16 .....":O .{H..O.
.... < 16 bytes >
00b0 94 90 a0 34 9a fa cf 71 33 19 65 f5 73 14 4e 93 ...4...q3.e.s.N.
.... < 8 bytes >
00c0 b2 89 a3 95 14 8e df e5 ........
Makes a total of a 34 byte sized fhandle.
...
> Network File System, FSINFO Call DH:0xb2ae682f
> [Program Version: 3]
> [V3 Procedure: FSINFO (19)]
> object
> length: 34
> [hash (CRC-32): 0xb2ae682f]
> [Name: 192.168.10.1:/gv0/fleming1/db0/ALTUS_data]
> filehandle:
> 3a4f20117b487f884f169490a0349afacf71e8bd0e2198c34cb88a0231dbeb071035
Bytes of the object parameter:
< length > < 6 bytes >
00b0 .. .. .. .. .. .. 00 00 00 22 3a 4f 20 11 7b 48 .........":O .{H
.... < 16 bytes >
00c0 7f 88 4f 16 94 90 a0 34 9a fa cf 71 e8 bd 0e 21 ..O....4...q...!
.... < 12 bytes > < ? >
00d0 98 c3 4c b8 8a 02 31 db eb 07 10 35 00 00 ..L...1....5..
Note the two 0-bytes at the end. Thinking about that, the following may
be the case:
XDR (http://tools.ietf.org/html/rfc4506, the encoding used for the RPC
protocol) uses 'blocks' for alignment. A fhandle byte array that is
34-bytes long, needs to be 34 / 4 + 1 = 36 bytes in size. The 'length'
given in the structure tells the consumer to ignore the two tailing
bytes.
The NFSv3 specification (http://tools.ietf.org/html/rfc1813#page-21)
defines the nfs_fh3 as a opaque (not bytes) structure.
> So for some reason, gluster is happy with Linux's request but not
> Oracle's.
My guess is that this (untested) change would fix it, can you try that?
--- a/rpc/xdr/src/xdr-nfs3.c
+++ b/rpc/xdr/src/xdr-nfs3.c
@@ -184,7 +184,7 @@ xdr_specdata3 (XDR *xdrs, specdata3 *objp)
bool_t
xdr_nfs_fh3 (XDR *xdrs, nfs_fh3 *objp)
{
- if (!xdr_bytes (xdrs, (char **)&objp->data.data_val, (u_int *) &objp->data.data_len, NFS3_FHSIZE))
+ if (!xdr_opaque (xdrs, &objp, (u_int *) &objp->data.data_len))
return FALSE;
return TRUE;
}
HTH,
Niels
>
> All I get out of gluster is:
> [2013-04-08 12:54:32.206312] E [nfs3.c:4741:nfs3svc_fsinfo] 0-nfs-nfsv3:
> Error decoding arguments
>
>
> I've attached abridged packet captures and text explanations of the
> packets (thanks to wireshark).
>
> Can someone please look at this and determine if it's gluster's parsing
> of the RPC call to blame, or if it's Oracle?
>
> This is the same setup on which I reported the NFS race condition bug.
> It does have that patch applied.
> Details:
> http://lists.gnu.org/archive/html/gluster-devel/2013-04/msg00014.html
>
> Thanks,
>
> Michael
>
> --
> Michael Brown | `One of the main causes of the fall of
> Systems Consultant | the Roman Empire was that, lacking zero,
> Net Direct Inc. | they had no way to indicate successful
> ?: +1 519 883 1172 x5106 | termination of their C programs.' - Firth
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel@xxxxxxxxxx
> https://lists.nongnu.org/mailman/listinfo/gluster-devel
--
Niels de Vos
Sr. Software Maintenance Engineer
Support Engineering Group
Red Hat Global Support Services
