> On Feb 17, 2016, at 11:34 AM, J. Bruce Fields <bfields@xxxxxxxxxxxx> wrote: > > On Wed, Feb 17, 2016 at 02:50:44PM +0000, Adamson, Andy wrote: >> >>> On Feb 17, 2016, at 5:31 AM, Martin Houry <martinhoury@xxxxxxxxx> wrote: >>> >>> ----- Original Message ----- >>> >>>> From: andros@xxxxxxxxxx >>>> To: "trond myklebust" <trond.myklebust@xxxxxxxxxxxxxxx> >>>> Cc: linux-nfs@xxxxxxxxxxxxxxx, "Andy Adamson" <andros@xxxxxxxxxx> >>>> Sent: Friday, February 5, 2016 4:08:32 PM >>>> Subject: [PATCH RFC Version 1 0/6] Request for Comment: NFS4.1 Session Trunking >>>> >>>> From: Andy Adamson <andros@xxxxxxxxxx> >>>> >>>> This early patch set implements NFSv4.1 session trunking utilizing Tronds >>>> multipath feature. This works against an upstream linux server with >>>> two NICs as the EXCHANGE_ID info is the same for both NICs. >>>> >>>> Unmodified code will create a new nfs_client structure for the mount as >>>> nfs_match_client fails due to the different full server address. >>>> Tunking is detected by comparing the eir_server_owner.major/minor/scope >>>> and the eir_server_impl_id to an existing mount. When trunking is detected, >>>> the 2nd mount point uses the existing (not new) nfs_client for the new >>>> mount point. >>>> >>>> When trunking is detected, this patch set adds an rpc_xprt for the new server >>>> address to the exisiting mount, calls BIND_CONN_TO_SESSION, and destroys >>>> the new clientid (no second mount). >>>> >>>> We then get round robin use of the two NICs off one mount point all >>>> using a single session. >>>> >>>> >>>> Is using mount a good way to setup NFSv4.1 session trunking? >>>> >>>> The first two patches fix issues with the multipath code. >>>> >>>> SUNRPC: allow rpc_xprt_switch_add_xprt to add xprts on the same net >>>> This patch adds a flag to allow multipathing within the origial net. >>>> I don't understand why this was not allowed, so I expect a different >>>> solution for this issue. >>>> >>>> >>>> Comments please. >>>> >>>> Thanks >>>> >>>> -->Andy >>>> >>>> Andy Adamson (6): >>>> SUNRPC fix rpc_clnt_add_xprt setup return >>>> SUNRPC fix rpc_clnt_xprt_iter_init infinite loop >>>> SUNRPC: allow rpc_xprt_switch_add_xprt to add xprts on the same net >>>> NFS session trunking support: add trunk xprt to nfs4.1 mount >>>> NFS add callback_ops to nfs4_proc_bind_conn_to_session_callback >>>> NFS schedule bind_conn_to_session for new NFSv4.1 transport >>>> >>>> fs/nfs/client.c | 2 ++ >>>> fs/nfs/nfs4client.c | 29 +++++++++++++++++++++++++++-- >>>> fs/nfs/nfs4proc.c | 10 ++++++++++ >>>> include/linux/nfs_fs_sb.h | 1 + >>>> include/linux/sunrpc/clnt.h | 1 + >>>> include/linux/sunrpc/xprtmultipath.h | 6 +++++- >>>> net/sunrpc/clnt.c | 20 +++++++++++++------- >>>> net/sunrpc/xprtmultipath.c | 35 >>>> +++++++++++++++++++++++++++++++++-- >>>> 8 files changed, 92 insertions(+), 12 deletions(-) >>>> >>>> -- >>>> 1.8.3.1 >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>> >>> Tried and approved! >> >> Thanks for testing. As Trond pointed out, the correct way to indicate multiple hostnames is on the mount command line >> >> mount -o minorversion=1 host1,host2,…,hostn:/<export> /<mntdir> >> >> I have a new patch set that is working and that I am testing and cleaning up that adds multiple hostname support to mount.nfs for NFSv4.1, and that kicks off an EXCHANGE_ID for each multi-host address. If the EXCHANGE_ID succeeds, a BIND_CONN_TO_SESSION is also called. > > I'm surprised there's BIND_CONN_TO_SESSION in the case Martin's > testing. I didn't think it was necessary unless you were using > kerberos. Maybe I'm forgetting some subtlety…. The first host (host1 above) is the mount host. Each additional host is a new connection. An EXCHANGE_ID is sent on the new connection, and the resultant serverscope, server owner, clientid, etc from the new connection EXCHANGE_ID reply is compared to the mount exchange_id args stored in the nfs_client struct. If it passes the session trunking requirements, then the connection needs to be bound to the session to enable session trunking. —>Andy RFC5661 2.10.5 …… Session Trunking. If the eia_clientowner argument is the same in two different EXCHANGE_ID requests, and the eir_clientid, eir_server_owner.so_major_id, eir_server_owner.so_minor_id, and eir_server_scope results match in both EXCHANGE_ID results, then the client is permitted to perform session trunking. If the client has no session mapping to the tuple of eir_clientid, eir_server_owner.so_major_id, eir_server_scope, and eir_server_owner.so_minor_id, then it creates the session via a CREATE_SESSION operation over one of the connections, which associates the connection to the session. If there is a session for the tuple, the client can send BIND_CONN_TO_SESSION to associate the connection to the session. Of course, if the client does not desire to use session trunking, it is not required to do so. It can invoke CREATE_SESSION on the connection. This will result in client ID trunking as described below. It can also decide to drop the connection if it does not choose to use trunking. > > --b. > >> >> —>Andy >> >> >>> >>> Here is the steps to make it works for those who wants to try it. : >>> >>> Test configuration : >>> >>> Running Server Kernel Version : 3.2.0 >>> Running Client Kernel Version : 3.2.0 >>> Patch deployment Client Kernel Version : 4.5.0 rc4+ >>> Debian 7.9 >>> GNS3 1.4.1 >>> Wireshark 1.12.3 >>> VirtualBox 5.0.14 >>> Client and Server are Virtualbox VM's conected each other with a GNS3 Switch >>> >>> NO AUTHENTIFICATION - this is only for test purpose >>> >>> +----------+ +----------+ >>> | | 192.168.1.2 /24 +----------+ 192.168.1.3 /24 | | >>> | +-----------------+ +-----------------+ | >>> | Client | | Switch | | Server | >>> | +-----------------+ +-----------------+ | >>> | | 192.168.2.20/24 +----------+ 192.168.2.30 /24| | >>> +----------+ +----------+ >>> >>> >>> Steps : >>> ------------------------------------------------------------------- >>> Server NFS4.1 configuration : >>> >>> Make sure you have an internet connection. >>> >>> Commands assuming you are root: >>> >>> apt-get update && apt-get upgrade >>> apt-get install nfs-kernel-server >>> mkdir home/testnfs >>> chmod 777 home/testnfs >>> nano etc/exports >>> #Add these line in the "exports" file to set the "testnfs" folder available >>> /home/testnfs 192.168.1.2(rw,sync) >>> /home/testnfs 192.168.2.20(rw,sync) >>> #end >>> >>> #Enable NFSv4.1 >>> etc/init.d/nfs-kernel-server stop >>> nano proc/fs/nfsd/versions >>> #set +2 +3 +4 -4.1 to +2 +3 +4 +4.1 >>> etc/init.d/nfs-kernel-server start >>> >>> #server ready >>> #end server configuration >>> ------------------------------------------------------------------- >>> Client NFS4.1 configuration : >>> >>> Commands assuming you are root: >>> #------------Prepare new Kernel with new patches---------- >>> apt-get install libncurses5-dev gcc make git exuberant-ctags bc libssl-dev >>> mkdir kernels >>> cd kernels >>> git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git >>> #it takes a while... >>> cd linux-stable >>> cp /boot/config-`uname -r`* .config >>> #the -j4 stand for 4 jobs running at the same time, ideal for >>> multi-core processor. >>> make -j4 >>> #It takes a while >>> >>> #--------------------Install the patches------------------ >>> apt-get install patch >>> cd / >>> mkdir patch >>> cd patch >>> mkdir rpcmultipathpatch >>> mkdir sessiontrunkingpatch >>> #Get the Trond Myklebust's patch : >>> http://www.spinics.net/lists/linux-nfs/msg56365.html >>> nano rpcmultipathpatch/patch1.patch >>> . >>> . >>> . >>> nano rpcmultipathpatch/patch13.patch >>> #Get the Andy Adamson's patches : >>> http://www.spinics.net/lists/linux-nfs/msg56437.html >>> nano sessiontrunkingpatch/patch1.patch >>> . >>> . >>> . >>> nano sessiontrunkingpatch/patch6.patch >>> >>> >>> cd / >>> >>> #Apply rpc multipath patch >>> patch -p1 < rpcmultipathpatch/patch1.patch >>> patch -p1 < rpcmultipathpatch/patch2.patch >>> patch -p1 < rpcmultipathpatch/patch3.patch >>> patch -p1 < rpcmultipathpatch/patch4.patch >>> patch -p1 < rpcmultipathpatch/patch5.patch >>> patch -p1 < rpcmultipathpatch/patch6.patch >>> patch -p1 < rpcmultipathpatch/patch7.patch >>> patch -p1 < rpcmultipathpatch/patch8.patch >>> patch -p1 < rpcmultipathpatch/patch9.patch >>> patch -p1 < rpcmultipathpatch/patch10.patch >>> patch -p1 < rpcmultipathpatch/patch11.patch >>> patch -p1 < rpcmultipathpatch/patch12.patch >>> patch -p1 < rpcmultipathpatch/patch13.patch >>> >>> #Apply client session trunking patches >>> patch -p1 < sessiontrunkingpatch/patch1.patch >>> patch -p1 < sessiontrunkingpatch/patch2.patch >>> patch -p1 < sessiontrunkingpatch/patch3.patch >>> patch -p1 < sessiontrunkingpatch/patch4.patch >>> patch -p1 < sessiontrunkingpatch/patch5.patch >>> patch -p1 < sessiontrunkingpatch/patch6.patch >>> >>> make -j4 >>> >>> #I got a compilation error in the "net/sunrpc/xprtmultipath.c" line 220 >>> #at WRITE_ONCE(&xpi->xpi_cursor,NULL); >>> #error: lvalue required as unary ‘&’ operand >>> #I remove the "&" >>> >>> make -j4 M=net/sunrpc >>> >>> make modules_install install >>> #reboot and boot on the new kernel (with GRUB) >>> >>> >>> #--------------------Mount-------------------- >>> mkdir testmount >>> mount -tnfs4 -ominorversion=1 192.168.1.3:/home/testnfs testmount >>> mount -tnfs4 -ominorversion=1 192.168.2.30:/home/testnfs testmount >>> >>> #end client configuration >>> >>> The round-robin implementation distribute correctly the operations >>> between the two interfaces and the BIND_CONN_TO_SESSION seems to work. >>> >>> >>> If you cut a cable, the client continue to send nfs packets on it, so >>> you loose one operation on two. >>> >>> >>> >>> >>> >>> Thanks to http://kernelnewbies.org/. >>> Thanks to Trond Myklebust and Andy Adamson for the patches >> >> N?????r??y????b?X??ǧv?^?){.n?+????{???"??^n?r???z???h?????&?? ?G???h?(?階?ݢj"???m??????z?ޖ???f???h???~?m ��.n��������+%������w��{.n�����{��w��� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
