On Thu, 30 Jan 2014 15:06:34 -0500 Steve Dickson <SteveD@xxxxxxxxxx> wrote:
>
>
> On 01/30/2014 01:24 AM, NeilBrown wrote:
> >
> > Hi all,
> >
> > I would really like to see a common set of systemd unit files used for
> > nfs-utils by every distribution (that actually uses systemd), and would like
> > those unit files to be in the upstream nfs-utils package.
> >
> > To that end I have put together the following collection of unit files which
> > seem right to me, and appear to work as I think I want them to work.
> >
> > I've looked at the unit files in Fedora and borrowed some ideas while
> > discarding and changing others. I won't try to list and justify all the
> > changes here, but I'm perfectly willing (possibly even eager) to justify
> > anything specific that anyone cares to ask about.
> >
> > So:
> > 1/ Do you agree that a collection of systemd unit files belongs in
> > nfs-utils?
> I would think so...
> > 2/ Do you think it reasonable to expect most (systemd using) distros to
> > use the one set? I will certainly try to ensure openSUSE does if
> > upstream accepts them.
> > 3/ Do you have any comments/question about those below?
> >
> >
> > Thanks,
> > NeilBrown
> >
> > diff --git a/systemd/README b/systemd/README
> > new file mode 100644
> > index 000000000000..f0fb68825499
> > --- /dev/null
> > +++ b/systemd/README
> > @@ -0,0 +1,50 @@
> > +
> > +Notes about systemd unit files for nfs-utils.
> > +
> > +The unit files provided here should be sufficient for systemd
> > +to manage all daemons and related services provides by nfs-utils.
> > +
> > +They do *not* include any unit files for separate services such as
> > +rpc.rquotad (in the 'quota' package) or rpcbind.
> > +
> > +There are 4 units that can be 'enabled' or 'disabled' by systemctl, or
> > +by a suitable 'preset' setting:
> > +
> > + nfs-server.target
> > + If enabled, nfs service is started together with dependencies
> > + such as mountd, statd, rpc.idmapd
> > +
> > + nfs-client.target
> > + If enabled, daemons needs for an nfs client are enabled.
> > + This does *not* include rpc.statd. the rpc-statd.service unit
> > + is started by /usr/sbin/start-statd which mount.nfs will run
> > + if statd is needed.
> > +
> > + nfs-secure.target
> > + If enabled, then rpc.gssd will be run when either -client or
> > + -server is started, and rpc.svcgssd will be run when -server
> > + is started
> > +
> > + nfs-blkmap.target
> > + If enabled, then blkmapd will be run when nfs-client.target is
> > + started.
> Do we really need all of these targets? Could we cut it down to
> two. nfs.target and nfs-secure.target?
>
Excellent question.
My thought was to err on the side of providing too many rather than too few.
If a distro wants to remove a particular choice it can use the "presets"
feature off systemd to make sure that target is always enabled.
But I think you might be questioning specific targets rather than the general
number of targets - I respond to that below.
> > +
> > +
> > +It is possible that we should have an nfs-statd.target which can
> > +selectively enable statd being stared by -server and sm-notify
> > +being started by -server or -client. That way it could be disabled
> > +completely on V4-only configurations. Currently statd is always
> > +started on the server and sm-notify is always run if server or
> > +client is enabled.
> > +
> > +Stopping nfs-server will also stop rpc.mountd, and rpc.svcgssd.
> > +It cannot stop rpc.statd or rpc.gssd as they may be in use by the
> > +client and systemd cannot specify is two-pronged reverse dependency.
> > +(i.e. stop this unit if none of these units are running)
> > +
> > +Distro specific commandline configuration can be provided by
> > +installing a script /usr/lib/systemd/scripts/nfs-utils_env.sh
> > +This should write /run/sysconfig/nfs-utils based on configuration
> > +information such as in /etc/sysconfig/nfs or /etc/defaults/nfs.
> > +It should write to a tmp file and rename to the target to
> > +avoid parallel units seeing incomplete copies of the file.
>
> > diff --git a/systemd/nfs-blkmap.service b/systemd/nfs-blkmap.service
> > new file mode 100644
> > index 000000000000..7319a88661cc
> > --- /dev/null
> > +++ b/systemd/nfs-blkmap.service
> > @@ -0,0 +1,11 @@
> > +[Unit]
> > +Description=pNFS block layout mapping daemon
> > +After=var-lib-nfs-rpc_pipefs.mount
> > +Requires=var-lib-nfs-rpc_pipefs.mount
> > +
> > +Requisite=nfs-blkmap.target
> > +After=nfs-blkmap.target
> > +
> > +[Service]
> > +Type=forking
> > +ExecStart=/usr/sbin/blkmapd
> Is this even supported anymore? Maybe we can just drop it???
No idea. I found it as I was looking around and assumed it should be started.
If it is no longer supported, how does NFS now find the block devices for
direct pnfs access?
>
>
> > diff --git a/systemd/nfs-blkmap.target b/systemd/nfs-blkmap.target
> > new file mode 100644
> > index 000000000000..fbcc111152ee
> > --- /dev/null
> > +++ b/systemd/nfs-blkmap.target
> > @@ -0,0 +1,8 @@
> > +[Unit]
> > +Description= PNFS blkmaping enablement.
> > +# If this target is enabled, then blkmapd will be started
> > +# as required. If it is not enabled it won't.
> > +
> > +[Install]
> > +WantedBy=remote-fs.target
> > +WantedBy=multi-user.target
> > \ No newline at end of file
>
> Again, why is a client target needed? Now that idmappings are
> stored in the key ring what needs to be started on the client?
rpc.gssd?
Also sm-notify needs to be run on a machine this is used as an NFS client.
This should happen even if no NFS filesystems are mounted at boot.
An 'nfs-client' target seems the natural place to put that requirement.
>
> > diff --git a/systemd/nfs-client.target b/systemd/nfs-client.target
> > new file mode 100644
> > index 000000000000..fa591354abf3
> > --- /dev/null
> > +++ b/systemd/nfs-client.target
> > @@ -0,0 +1,13 @@
> > +[Unit]
> > +Description=NFS client services
> > +Before=remote-fs-pre.target
> > +Wants=remote-fs-pre.target
> > +
> > +# Note: we don't "Wants=rpc-statd.service" as "mount.nfs" will arrange to
> > +# start that on demand if needed.
> > +Wants=rpc-gssd.service nfs-blkmap.service rpc-statd-notify.service
> > +Before=rpc-gssd.service nfs-blkmap.service
> > +
> > +[Install]
> > +WantedBy=multi-user.target
> > +WantedBy=remote-fs.target
>
>
> > diff --git a/systemd/nfs-idmapd.service b/systemd/nfs-idmapd.service
> > new file mode 100644
> > index 000000000000..6c2e1537f064
> > --- /dev/null
> > +++ b/systemd/nfs-idmapd.service
> > @@ -0,0 +1,9 @@
> > +[Unit]
> > +Description=NFSv4 ID-name mapping service
> Fedora has in the [Unit]:
> BindTo=nfs-server.service
> After=nfs-server.service
> I guess I thought they were needed at the time..
Yes, I saw that. So I looked into it.
Apart from being a typo ("BindsTo" with an 's' is correct),
BindsTo has an extra effect if the target unit - nfs-server.service in this
case - disappears unexpectedly. e.g. if the process dies.
However nfs-server.service doesn't have a process which can die. Rather it
starts some kernel threads. I'm pretty sure systemd won't be able to link
those threads with nfs-server.service.
So nfs-server.service can never "die", so "BindsTo" add nothing useful.
"After" just seems wrong. The moment nfsd is started a request could come in
which could require an idmap lookup, so nfs-idmapd should already be there.
>
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=-/usr/lib/systemd/scritps/nfs-utils_env.sh
> > +
> > +Type=forking
> > +ExecStart=/usr/sbin/rpc.idmapd $RPCIDMAPDARGS
>
>
> > diff --git a/systemd/nfs-mountd.service b/systemd/nfs-mountd.service
> > new file mode 100644
> > index 000000000000..92e05ca309ee
> > --- /dev/null
> > +++ b/systemd/nfs-mountd.service
> > @@ -0,0 +1,13 @@
> > +[Unit]
> > +Description=NFS Mount Daemon
> > +Requires=proc-fs-nfsd.mount
> > +After=proc-fs-nfsd.mount
> > +After=network.target
> > +PartOf=nfs-server.service
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=-/usr/lib/systemd/scritps/nfs-utils_env.sh
> How does this script know who is calling it and what it
> has to do?
Its task is to read some config file like /etc/sysconfig/nfs and determine
what the command line arguments for each nfs program should be and write out
FOO_ARGS=
lines to /run/sysconfig/nfs-utils for each FOO which is an nfs program.
I have since thought this would work better as a separate unit which creates
the nfs-utils env file once, and have all the other units have Wants/After
dependencies.
>
> > +
> > +Type=forking
> > +ExecStart=/usr/sbin/rpc.mountd $RPCMOUNTDARGS
>
>
> > diff --git a/systemd/nfs-secure.target b/systemd/nfs-secure.target
> > new file mode 100644
> > index 000000000000..0127fdb07dbd
> > --- /dev/null
> > +++ b/systemd/nfs-secure.target
> > @@ -0,0 +1,8 @@
> > +[Unit]
> > +Description=Secure NFS client/server services
> > +# If this target is enabled, then rpc.gssd and rpc.svcgssd will be started
> > +# as required. If it is not enabled they won't.
> So the "Requisite=nfs-secure.target" in rpc-gssd.service cause the
> daemon to started?
This is a bit subtle.
The "Requisite=nfs-secure.target" in rpc-gssd.service means that service is
only allowed to start if this unit (nfs-secure.target) is already started.
So if nfs-secure.target is enabled, then rpc-gssd.service will start when
anything Wants it. If nfs-secure.target is not enabled, then
rpc-gssd.service will not start, even if something Wants it.
So enabling nfs-secure.target doesn't start anything, just allows a couple of
things to start if they are Wanted.
>
> > +
> > +[Install]
> > +WantedBy=remote-fs.target
> > +WantedBy=multi-user.target
> > \ No newline at end of file
>
>
> > diff --git a/systemd/nfs-server.service b/systemd/nfs-server.service
> > new file mode 100644
> > index 000000000000..9812866c66aa
> > --- /dev/null
> > +++ b/systemd/nfs-server.service
> > @@ -0,0 +1,24 @@
> > +[Unit]
> > +Description=NFS server
> > +DefaultDependencies=no
> > +Requires= network.target proc-fs-nfsd.mount rpcbind.target
> > +PartOf=nfs-server.target
> > +
> > +After= network.target proc-fs-nfsd.mount rpcbind.target nfs-mountd.service
> > +After= nfs-idmapd.service rpc-statd.service
> > +After= rpc-gssd.service rpc-svcgssd.service
> > +Before= rpc-statd-notify.service
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=-/usr/lib/systemd/scritps/nfs-utils_env.sh
> > +
> > +Type=oneshot
> > +RemainAfterExit=yes
> > +ExecStartPre=/usr/sbin/exportfs -r
> > +ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS
> Having ExecStartPre and ExecStartPost scripts gives
> us a place to do some pre and post server configuration.
>
> Granted it has not been needed in a while but I'm thinking
> it could come in handing...
systemd supports so-called "dropins". If a distro wants to add an
ExecStartPre for some distro-specific reason they can put it in
/etc/systemd/system/nfs-server.service.d/04-Fedora-hacks.conf
and it will be understood by systemd to be part of this service.
Of course if they were not distro-specific reasons we would include them in
the upstream package :-)
I note that the Fedora nfs-server.service has a preconfig
echo "$NFSD_V4_GRACE" > /proc/fs/nfsd/nfsv4gracetime
and a postconfig
/sbin/modprobe svcrdma
echo "rdma $RDMA_PORT" > /proc/fs/nfsd/portlist
These seem reasonably general, though I'm wondering why the RDMA port is set
in postconfig rather than preconfig.
Do you know why that is?
>
> > +ExecStop=/usr/sbin/rpc.nfsd 0
> > +ExecStopPost=/usr/sbin/exportfs -au
> > +ExecStopPost=/usr/sbin/exportfs -f
> > +
> > +ExecReload=/usr/sbin/exportfs -r
>
>
> > diff --git a/systemd/nfs-server.target b/systemd/nfs-server.target
> > new file mode 100644
> > index 000000000000..a3e629f022a9
> > --- /dev/null
> > +++ b/systemd/nfs-server.target
> > @@ -0,0 +1,8 @@
> > +[Unit]
> > +Description=NFS server services
> > +Requires=nfs-server.service nfs-mountd.service
> > +Wants=rpc-statd.service nfs-idmapd.service rpc-gssd.service rpc-svcgssd.service
> > +Wants=rpc-statd-notify.service
> > +
> > +[Install]
> > +WantedBy=multi-user.target
>
>
> > diff --git a/systemd/proc-fs-nfsd.mount b/systemd/proc-fs-nfsd.mount
> > new file mode 100644
> > index 000000000000..f44d52f3d67b
> > --- /dev/null
> > +++ b/systemd/proc-fs-nfsd.mount
> > @@ -0,0 +1,8 @@
> > +[Unit]
> > +Description=NFSD configuration filesystem
> > +DefaultDependencies=no
> > +
> > +[Mount]
> > +What=nfsd
> > +Where=/proc/fs/nfsd
> > +Type=nfsd
>
>
> > diff --git a/systemd/rpc-gssd.service b/systemd/rpc-gssd.service
> > new file mode 100644
> > index 000000000000..f0fef007d480
> > --- /dev/null
> > +++ b/systemd/rpc-gssd.service
> > @@ -0,0 +1,14 @@
> > +[Unit]
> > +Description=RPC security service for NFS client and server
> > +Requires=var-lib-nfs-rpc_pipefs.mount
> > +After=var-lib-nfs-rpc_pipefs.mount
> > +
> > +Requisite=nfs-secure.target
> > +After=nfs-secure.target
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=-/usr/lib/systemd/scritps/nfs-utils_env.sh
> > +
> > +Type=forking
> > +ExecStart=/usr/sbin/rpc.gssd $GSSDARGS
>
>
> > diff --git a/systemd/rpc-statd-notify.service b/systemd/rpc-statd-notify.service
> > new file mode 100644
> > index 000000000000..9d972fc7753a
> > --- /dev/null
> > +++ b/systemd/rpc-statd-notify.service
> > @@ -0,0 +1,17 @@
> > +[Unit]
> > +Description=Notify NFS peers of a restart
> > +DefaultDependencies=no
> > +Requires=network-online.target
> > +After=network-online.target nss-lookup.target
> > +
> > +# if we run an nfs server, it needs to be running before we
> > +# tell clients that it has restarted.
> > +After=nfs-server.service
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=/usr/lib/systemd/scritps/nfs-utils_env.sh
> > +
> > +Type=oneshot
> > +RemainAfterExit=yes
> > +ExecStart=-/usr/sbin/sm-notify -d $SMNOTIFYARGS
>
>
> > diff --git a/systemd/rpc-statd.service b/systemd/rpc-statd.service
> > new file mode 100644
> > index 000000000000..04962e542fbc
> > --- /dev/null
> > +++ b/systemd/rpc-statd.service
> > @@ -0,0 +1,12 @@
> > +[Unit]
> > +Description=NFS status monitor for NFSv2/3 locking.
> > +DefaultDependencies=no
> > +Requires=nss-lookup.target rpcbind.target
> > +After=network.target nss-lookup.target rpcbind.target
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=-/usr/lib/systemd/scritps/nfs-utils_env.sh
> > +
> > +Type=forking
> > +ExecStart=/usr/sbin/rpc.statd --no-notify $STATDARGS
>
>
> > diff --git a/systemd/rpc-svcgssd.service b/systemd/rpc-svcgssd.service
> > new file mode 100644
> > index 000000000000..f024d40a8f41
> > --- /dev/null
> > +++ b/systemd/rpc-svcgssd.service
> > @@ -0,0 +1,15 @@
> > +[Unit]
> > +Description=RPC security service for NFS server
> > +Requires=var-lib-nfs-rpc_pipefs.mount
> > +After=var-lib-nfs-rpc_pipefs.mount
> > +PartOf=nfs-server.service
> > +
> > +Requisite=nfs-secure.target
> > +After=nfs-secure.target
> > +
> > +[Service]
> > +EnvironmentFile=-/run/sysconfig/nfs-utils
> > +ExecStartPre=-/usr/lib/systemd/scritps/nfs-utils_env.sh
> > +
> > +Type=forking
> > +ExecStart=/usr/sbin/rpc.svcgssd $SVCGSSDARGS
>
>
> > diff --git a/systemd/var-lib-nfs-rpc_pipefs.mount b/systemd/var-lib-nfs-rpc_pipefs.mount
> > new file mode 100644
> > index 000000000000..cd614cf49f00
> > --- /dev/null
> > +++ b/systemd/var-lib-nfs-rpc_pipefs.mount
> > @@ -0,0 +1,8 @@
> > +[Unit]
> > +Description=RPC Pipe File System
> > +DefaultDependencies=no
> > +
> > +[Mount]
> > +What=sunrpc
> > +Where=/var/lib/nfs/rpc_pipefs
> > +Type=rpc_pipefs
>
>
> > diff --git a/utils/statd/start-statd b/utils/statd/start-statd
> > index 1b345a547932..cde3583238e3 100644
> > --- a/utils/statd/start-statd
> > +++ b/utils/statd/start-statd
> > @@ -5,5 +5,8 @@
> > # It should run statd with whatever flags are apropriate for this
> > # site.
> > PATH=/sbin:/usr/sbin
> > -exec rpc.statd --no-notify
> > -
> > +if systemctl start statd.service
> > +then :
> > +else
> > + exec rpc.statd --no-notify
> > +fi
> >
>
> How does lockd get started and configured?
Lockd is automatically started by the kernel (nfs and nfsd) when needed.
I noticed that Fedora had some scripts to optionally set the port numbers
that lockd would use. I left that out because I hoped we could find a better
way.
If lockd is compiled in to the kernel, then I think fs.nfs.nlm_XXXport should
be set via /etc/sysctl.conf.
If it is a module, then the module parameters should be configured via
an /etc/modprobe.d/lockd.conf file.
I wish this was unified somehow (should modprobe call sysctl??) but it isn't.
I appreciate that in neither case is it easy to get values out of a
'sysconfig' file in to the right place.
I cannot really think of a better solution than an 'nfs-lock' service which
sets these values, so I'll probably add it.
>
> Overall it looks reasonable... But this is a change
> of ABI... so it will be painful... :-(
What exactly is changing that will be painful? I would certainly like to
arrange the unit files to minimise your pain as much as possible, as long as
it doesn't compromise effectiveness.
Thanks for your review!
NeilBrown
Attachment:
signature.asc
Description: PGP signature
