On Wed, 2012-06-06 at 11:44 +0400, Michael Tokarev wrote:
> On 06.06.2012 08:12, Ian Kent wrote:
> > On Tue, 2012-06-05 at 10:26 +0400, Michael Tokarev wrote:
> []
> >> In order to eliminate this timeout, the socket used to contact
> >> the remote portmapper needs to be connected - this way, the
> >> OS will report ECONNREFUSED error instantly and the mount
> >> operation will proceed without delays even if the remote
> >> portmapper is not available.
> >
> > Right, but the recent changes have changed the order to TCP then UDP so
> > I'm not sure this is still accurate now, although I may not have changed
> > them all, I need to check that.
> >
> > But I thought ECONNREFUSED meant just that, in which case UDP should be
> > tried following that, in case only the TCP port is blocked.
>
> I tried exactly with the very recent changes. My test version
> is 09d4edb2469f9415f6ca33760d83b3c85b517de7, "autofs-5.0.6 -
> fix libtirpc name clash".
>
> Here's the debug output (with a few fprintf's I added):
>
> Starting automounter version 5.0.6, master map /etc/auto.master
> using kernel protocol version 5.02
> lookup_nss_read_master: reading master file /etc/auto.master
> parse_init: parse(sun): init gathered global options: (null)
> spawn_mount: mtab link detected, passing -n to mount
> spawn_umount: mtab link detected, passing -n to mount
> lookup_read_master: lookup(file): read entry /misc
> lookup_read_master: lookup(file): read entry +dir:/etc/auto.master.d
> lookup_nss_read_master: reading master dir /etc/auto.master.d
> lookup(dir): dir map /etc/auto.master.d missing or not readable
> lookup(file): failed to read included master map dir:/etc/auto.master.d
> master_do_mount: mounting /misc
> automount_path_to_fifo: fifo name /run/autofs.fifo-misc
> lookup_nss_read_map: reading map file /etc/auto.misc
> parse_init: parse(sun): init gathered global options: (null)
> spawn_mount: mtab link detected, passing -n to mount
> spawn_umount: mtab link detected, passing -n to mount
> mounted indirect on /misc with timeout 300, freq 75 seconds
> st_ready: st_ready(): state = 0 path /misc
> handle_packet: type = 3
> handle_packet_missing_indirect: token 101, name test, request pid 14024
> attempting to mount entry /misc/test
> lookup_mount: lookup(file): looking up test
> lookup_mount: lookup(file): test -> -vers=4 gnome:/data
> parse_mount: parse(sun): expanded entry: -vers=4 gnome:/data
> parse_mount: parse(sun): gathered options: vers=4
> parse_mount: parse(sun): dequote("gnome:/data") -> gnome:/data
> parse_mount: parse(sun): core of entry: options=vers=4, loc=gnome:/data
Oh right, I've resisted interfering with or even looking at the mount
options but I probably should look for the nfsvers= (and the
compatibility option vers=) option.
At the moment to tell autofs your expecting nfsv4 only you need to use
the pseudo option, fstype, aka fstype=nfs4.
If you don't do this then the system default is used which means it's
probably nfs4 these days, with automatic fallback to v3 or v2. So
consulting the port mapping name service is the right thing to do.
Note that for fallback to work you need to be using sane NFS servers,
ones that don't have the use of a global root as the only way to export
the file systems.
> sun_mount: parse(sun): mounting root /misc, mountpoint test, what gnome:/data, fstype nfs, options vers=4
> mount_mount: mount(nfs): root=/misc name=test what=gnome:/data, fstype=nfs, options=vers=4
> mount_mount: mount(nfs): nfs options="vers=4", nobind=0, nosymlink=0, ro=0
> get_nfs_info: called with host gnome(192.168.88.4) proto tcp version 0x70
> get_nfs_info: nfs v4 rpc ping time: 0.004577
> get_nfs_info.portmap_getclient begin
> rpc_portmap_getclient.create_client begin
> rpc_portmap_getclient.create_client end
> get_nfs_info.portmap_getclient end
> get_nfs_info: host gnome cost 4576 weight 0
> get_nfs_info: done
> get_nfs_info: called with host gnome(192.168.88.4) proto udp version 0x70
> get_nfs_info: nfs v4 rpc ping time: 0.000387
> get_nfs_info.portmap_getclient begin
> rpc_portmap_getclient.create_client begin
> rpc_portmap_getclient.create_client end
> get_nfs_info.portmap_getclient end
> get_nfs_info.portmap_getport begin
> rpc_portmap_getport.clnt_call begin
> [several seconds pause here]
> rpc_portmap_getport.clnt_call end
> get_nfs_info.portmap_getport end
> get_nfs_info: host gnome cost 386 weight 0
> get_nfs_info: done
> prune_host_list: selected subset of hosts that support NFS4 over TCP
> mount_mount: mount(nfs): calling mkdir_path /misc/test
> mount_mount: mount(nfs): calling mount -t nfs -s -o vers=4 gnome:/data /misc/test
> spawn_mount: mtab link detected, passing -n to mount
> mount_mount: mount(nfs): mounted gnome:/data on /misc/test
> dev_ioctl_send_ready: token = 101
> mounted /misc/test
>
> It is doing first clnt_call() in rpc_portmap_getport(),
> which is PMAPPROC_NULL, over UDP which times out. After
> that it just mounts things.
>
> >> This is rpc_portmap_getport() function.
> >>
> >> I tried to follow the code, but the logic there is so twisted
> >> I've lost track of things quite quickly.
> >
> > Yes, it is a bit convoluted.
> >
> > This code was originally written for two reasons.
> > First to take control over the timeouts to avoid lengthy waits and
> > second to reduce socket usage by allowing callers to reuse socket file
> > descriptors.
>
> Is that okay to made these sockets connected? I mean, is the
> same socket can be reused to connect to different destination
> (host or port)?
>
> >> The question is: is it possible to use connected UDP sockets
> >> there in all cases, or is conn_info->client reused for
> >> something else? If it is, I guess we may just get rid of
> >> cached (UDP) client and create new client for each call.
> >
> > Don't think so.
>
> >> Or is this completely backwards? I mean, UDP is really
> >> unreliable protocol, for it to actually work the app
> >> should retry sending packet in case of timeout, and the
> >> protocol is easy to spoof too...
> >
> > Backward in the sense that we probably should (and should already) be
> > using TCP first.
>
> If you mean TCP for the portmapper access, well, maybe it
> actually does, but obviously the portmapper does not respond
> there if it isn't even running on the remote host.
>
> Even with TCP, in case the portmapper port is firewalled,
> we may get timeout. But this should be easy to work around
> in the configuration -- when port is specified explicltly,
> portmapper isn't consulted at all.
It's not quite that simple for the case below.
Exactly what your asking and what your trying to solve is not clear.
Above you quote what looks like a simple indirect map mount request but
below you're talking about the internal hosts map (I think) which is a
completely different story.
We need to clarify exactly what issue we are talking about and what we
might do to resolve it since there may be more than one and discussions
like this can quickly become completely confusing.
So, let's take a little care to make sure we understand fully the issues
and try to identify them individually as we go.
>
> > Another thing is that, since the change to mount.nfs(8), we now also
> > need to change to probing on simple automounts (as well as multi-mounts)
> > in order to maintain a reasonable interactive response.
>
> I need to write another lengthly email about this. I think
> the whole approach to checking for /sbin/modprobe and /proc
> (neither of which is actually used in the code), /sbin/mount.nfs,
> fsck.ext* at configure time is wrong: instead, you should
> assume nothing at compile time and call these programs when
> needed at runtime using either fixed - standard paths or
> searching in $PATH, probing their existance if necessary.
>
> It turned out that Debian autofs package has never used
> the "new" nfs mount code at all, because during compile
> time mount.nfs has never been available (the compile is
> done in a chroot), so the code never checked the "symbolic"
> mount.nfs argument and always used the fallback.
Yes, another day.
>
> But that's for another email. For now I just want to
> understand what to do with this UDP portmap probing,
> the trivial fix for the problem at hand (assuming the
> portmap port isn't firewalled) is to make UDP socket
> connected, so ECONNREFUSED for UDP will be noticed
> immediately without resorting to timeout.
I think the trivial fix for simple mounts like the one above is to
specify the fstype=nfs4 option and set MOUNT_WAIT, in the autofs
configuration, to some sensible value for the site in question.
The fstype option will tell autofs you want only nfs4 mounts and pass
"-t nfs4" to mount(8) so that it doesn't attempt to fall back to v3 etc.
and the MOUNT_WAIT setting will tell autofs not to probe when performing
simple mounts. This was done to provide a way to use the previous autofs
mechanism without having to endure lengthy timeouts. Using MOUNT_WAIT
isn't ideal but that's a discussion for another day.
>
> This will keep the general potential problem of using
> UDP in the first place, and this might want a separate
> solution (maybe eliminating UDP altogether), but I'm
> not sure I want to change that much in one go :)
>
> >> Alternatively, we may try to contact 2049 port first,
> >> and if that one does not respond, try to contact the
> >> portmapper. But I'm not sure if this is right thing
> >> to do to start with.
> >
> > Don't know that yet either.
>
> Currently it doesn't work with at all /net map. I asked
> several times on IRC about this very case, -- whenever it
> is ever possible to enumerate (nfsv4) exports without
> portmapper, but no one replied :( Note: I know right to
> nothing about this RPC internals.
Since the recommended way to use a hosts map is by using the internal
autofs hosts map I'll talk about that. If the customer is using the
example program map auto.net then they can work out how they wish to do
it themselves.
The internal hosts map gets the list of exports from the nfs server and
constructs a multi-mount map entry from that.
To do that it must consult rpc.mountd(8) on the server which means it
must consult the port mapping service to get the port it's available on.
So that's one reason the hosts map is having a problem. It wouldn't too
hard to add a configuration option for the mountd port and update the
hosts map code to behave itself when the fstype=nfs4 option is also
present. Clearly the mountd service might not be running on the same
port on all servers but we're talking about firewalls which should be
able to do port redirects based on ip address. In any case that's not
really an autofs problem.
The next problem is that rpc.mountd(8) provides limited information. We
only get the export path and access list. There is no information about
nfs protocol version. There's also no way to tell if the exports use the
global root. If the server exports are using a global root there's no
way to know what the base path of the global root is. That means autofs
has no (reliable) way to find out what path to use to alter the path
used in mounts to the server.
Essentially, nfs setups that use the global root only are not supported
by the internal hosts map.
>
> Thanks,
>
> /mjt
--
To unsubscribe from this list: send the line "unsubscribe autofs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
