Hi list,
I have a situation where I need to run a command and wait for its
completion before unmounting a file system (/enc in my case). My problem
is that systemd sometimes waits for the completion of the command, and
sometimes doesn't.
So the setup is:
* /enc is a mounted encrypted (luks) volume
* /var/lib/mysql is a bind mount to /enc/mysql
* MariaDB is using /var/lib/mysql
* We've set up a "requires" and "after" dependency chain from MariaDB
all the way to the luks volume. This works 100% reliably during startup,
even if something takes too long, fails, or if we have to manually
fiddle with stuff.
* Ubuntu 20.04 with systemd 245.4
The unit of the command that needs to run before unmounting /enc looks
like this:
[Unit]
Description=server-secrets-prepare-reboot-enc service
After=network-online.target enc.mount
Requires=network-online.target
BindsTo=enc.mount
[Service]
ExecStart=/bin/true
ExecStop=/usr/local/sbin/server-secrets reboot "/enc"
Restart=no
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=server-secrets-prepare-reboot-enc
User=root
Type=oneshot
RemainAfterExit=yes
[Install]
WantedBy=enc.mount
I've added a 10s sleep to the "server-secrets" command in order to
eliminate "works by chance" situations.
In normal circumstances everything works perfectly. I can reboot the
server and systemd waits >10s with unmounting /enc until server-secrets
is done. I can stop mariadb, umount /var/lib/mysql and /enc, and systemd
runs server-secrets just fine. I can re-mount /enc, unmount it again,
everything works.
Where things start to fall apart is when I reboot the server ("reboot"
or "shutdown -r now") after manually stopping MariaDB. In this case
systemd starts the server-secrets command before unmounting /enc, but
does not wait for its completion, and immediately unmounts /enc, causing
the command to fail.
Note that manually stopping MariaDB does not unmount anything, it
just... stops MariaDB. But somehow it causes systemd to change its
ordering behavior during shutdown.
Does this ring any bells?
Thank you!
Best regards,
David Gubler
--
David Gubler
System Engineer
VSHN AG | Neugasse 10 | CH-8005 Zürich
+41 44 545 53 00 | https://vshn.ch
