> On May 26, 2016, at 10:25 AM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: > > >> On May 26, 2016, at 9:54 AM, Andy Adamson <androsadamson@xxxxxxxxx> wrote: >> >> On Wed, May 25, 2016 at 2:55 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >>> >>>> On May 25, 2016, at 2:48 PM, bfields@xxxxxxxxxxxx wrote: >>>> >>>> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote: >>>>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs. >>>>> >>>>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list. >>>>> >>>>> Some background: >>>>> >>>>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10): >>>>> >>>>> The pseudo-fs GETATTR(fs_locations) probe session trunking >>>>> was tested against a Linux server with a pseudo-fs >>>>> export stanza (e.g. a stanza with the fsid=0 or fsid=root >>>>> export option) and a replicas= export option >>>>> (replicas=<path1>@<server1>:<path2>@<server2>..) >>>>> Note that this configuration is for testing only. A future >>>>> patchset will add the replicas= configuration to the >>>>> NFSEXP_V4ROOT nfsd and mountd processing. >>>>> >>>>> >>>>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline. >>>>> >>>>> >>>>>> Begin forwarded message: >>>>>> >>>>>> From: Chuck Lever <chuck.lever@xxxxxxxxxx> >>>>>> Subject: Re: Configuring fs_locations on Linux upstream server >>>>>> Date: May 6, 2016 at 4:31:00 PM EDT >>>>>> To: "J. Bruce Fields" <bfields@xxxxxxxxxxxx> >>>>>> Cc: "Adamson, Andy" <William.Adamson@xxxxxxxxxx> >>>>>> >>>>>> >>>>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@xxxxxxxxxxxx> wrote: >>>>>>> >>>>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote: >>>>>>>> Seems like when a server does not return a list, that is >>>>>>>> information the client can use: basically, there is no >>>>>>>> ability to do any session trunking. It has to be set up >>>>>>>> explicitly; is that a bad thing, operationally? >>>>>>> >>>>>>> I like the idea of it being opt in on the server. >>>>>>> >>>>>>> Suppose the server transparently starts advertising all available >>>>>>> addresses for session trunking. It's not hard to imagine cases where >>>>>>> that would go wrong. E.g., maybe the server has the odd wireless or >>>>>>> 100Mb or other interface that happens to work but that's slow. Then >>>>>>> somebody upgrades their server and performance goes down and it may take >>>>>>> them a while to figure out why. Whereas if they'd had to opt in they'd >>>>>>> probably have avoided advertising an inappropriate interface. Or at >>>>>>> least they'd have a better chance of figuring out that turning on >>>>>>> trunking was what caused the problem. >>>>>>> >>>>>>> I'd rather not force people to export "/" explicitly, though. It's fine >>>>>>> for testing, but: >>>>>>> >>>>>>> - I don't think we give a way to do an explicit V4ROOT export, >>>>>>> so they'd be exposing their entire root partition. We could >>>>>>> fix that, but >>>>>>> - the pseudofs just seems to me like something people shouldn't >>>>>>> normally have to think about. It's a protocol implementation >>>>>>> detail, I'd rather hide it. It'd be to easy to configure it a >>>>>>> little wrong, I think. >>>>>>> >>>>>>> We can still do this by adding a replicas= option to the / export, but >>>>>>> we can let rpc.mountd do that internally instead of making the admin add >>>>>>> it to /etc/exports. >>>>>>> >>>>>>> But then you still need a way for the admin to tell rpc.mountd to cook >>>>>>> up the replicas= option..... I'm not sure what that should look like. >>>>> >>>>> Idea 1: extra syntax in /etc/exports >>>> >>>> It's not really export-specific information. I wonder if it'd be better >>>> to pass it on the rpc.nfsd commandline? >>>> >>>> rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2" >>>> >>>> (and then that can be configured in /etc/sysconfig/nfs or whatever)? >> >> Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the >> preferred way? > > I don't prefer it. > > See below: I think we want something that is more > convenient to update automatically. Fine, but I’m having difficulty in understanding the design you are suggesting to fulfill the update automatically requirement. See inline below. > > >> Is /etc/sysconfig/nfs read upon reboot? > > It's read by all the start-up scripts related to NFS. > > >> -->Andy >> >> >> >>>> >>>>>>> Maybe some extra syntax in /etc/exports, but what do they need to give >>>>>>> us--just one list of IP addresses? Chuck, any ideas? >>>>> >>>>> Idea 2: xattr attached to “/" >>>>> >>>>>> >>>>>> How about using the same approach used for junctions: >>>>>> put the list in an xattr attached to / ? mountd can >>>>>> extract that when the kernel asks for help satisfying >>>>>> a GETATTR(fs_locations) on V4ROOT. >>>> >>>> I don't think that works. "/" isn't a good place to put configuration. >>>> It could be read-only, among other things. >>>> >>>>> Idea 3: new /etc/ config file >>>>>> >>>>>> Or it could be put in a separate config file in /etc. >>>>>> You might want to specify more than just the i/f list >>>>>> here; for instance, the security policy for the >>>>>> pseudofs, or a constant fsid UUID, among other things. >>>>> >>>>> >>>>> API to update the i/f list. This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system. >>>>> >>>>>> >>>>>> Also, I suggested to Andy earlier: >>>>>> >>>>>>> I find myself leaning towards mechanisms that are easy >>>>>>> both for admins and for programs (ie, an API). Perhaps >>>>>>> one day you might want to add a command that updates the >>>>>>> i/f list from the scripts in /etc/sysconfig/network-scripts, >>>>>>> for instance. >>>>>>> >>>>>>> As part of an ifup: >>>>>>> >>>>>>> nfspfs add <addr> >>>>>>> >>>>>>> and ifdown: >>>>>>> >>>>>>> nfspfs remove <addr> >>>>>>> >>>>>>> I wrote some Python code to manipulate entries in >>>>>>> /etc/exports, now found in fedfs-utils. It's icky. >>>>>> >>>>>> I think we should move away from "edit this file >>>>>> and save it, then restart rpc.xyzpdq". Build some >>>>>> command line interfaces for this. >>>> >>>> I'm OK with that. >>>> >>>> (Note do have that for information in /etc/exports--we have exportfs. >>>> Is there a reason that didn't work for fedfs-utils?) >>> >>> To make changes that can survive a server reboot, >>> you have to update /etc/exports. Your suggestion then is to build a new command-line interface to: - tell mountd of a V4ROOT multipath list? - have said list survive reboots, e.g. stored in a file? Please povide more detail on your thoughts. Thanks —> Andy >>> >>> >>> >>>> --b. >>>> >>>>>> >>>>>> And as you have suggested many times: separate >>>>>> policy from mechanism. /etc/exports is the >>>>>> mechanism. >>>>>> >>>>>> -- >>>>>> Chuck Lever >>>>> >>>>> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?) >>>>> >>>>> Thanks >>>>> >>>>> —>Andy >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> -- >>> Chuck Lever >>> >>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > Chuck Lever > > > ��.n��������+%������w��{.n�����{��w��� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
