Running a service *just* before unmounting filesystems

[lennart@xxxxxxxxxxxxxx (Lennart Poettering)]

On Mo, 11.06.18 15:37, Hans de Goede (hdegoede at redhat.com) wrote:

> > Uurks. Quite frankly, it appears strange to me to delay this for this
> > long. I mean we reworked most code that delayed worked to shutdown
> > like this these days to happen as early as possible to make sure we
> > don't lose state unnecessarily. For example the RTC syncing is
> > generally done when the RTC is changed instead of synced back during
> > shutdown. Hence, why not simply write this out when the boot is
> > successful?
> 
> There are 2 flags / grub environment variables in play here:
> 
> boot_success
> shutdown_success
> 
> The idea being that we also want to show the grub menu if the
> system did not shutdown cleanly (or somewhat cleanly given
> that by the time we know we really have a clean shutdown we
> can no longer write the grubenv).

Well, speaking from the receiving end of the bug report hose, I can
tell you that shutdown hangs are almost exclusively happening way
after we regularly unmount directories such as /efi or
/boot. i.e. such hangs happen in the second phase of shutdown where we
clean-up everything that couldn't be cleaned up during normal unit
shutdown. (Or in fact even later, after the system returned back into
the initrd.)

It am very sure it's not worth trying to maintain a shutdown_sucess
variable that is determined that early. That's a pointless excercise,
you won't catch 99% of relevant issues that way...

> The feedback I've been getting on the fedora-devel list is that
> people are somewhat worried about not being able to get to the
> grub menu, so we are being very careful here and err-ing on the
> side of showing the menu too often, rather then possibly not
> often enough.

sd-boot solves that by always showing the menu if any key is pressed
while sd-boot initializes. This means you can hold down any key you
like during early boot and the menu is guaranteed to be shown. Why not
do that in grub, too?

I mean, if you are looking for a reliable way to get the menu back if
things are bad, then such a shutdown hook is not going to help you,
it's not useful to write out shutdown sucess info so early...

> > Note that /boot or /efi is very likely an automount point, (that's at
> > least how we recommend things to be set up, as this provides the best
> > guarantees that the ESP remains is a clean state, as it will be very
> > quickly after the last access, and hence only be in dirty state during
> > a very short timeframe around accesses), and in that case "right
> > before unmount" doesn't make much sense in general, as that would be
> > pretty much all the time (that said, I don't think fedora/Anaconda
> > makes use automount points for /boot and /esp, or even systemd's
> > auto-discovery of the ESP currently, they haven't seen the light there
> > yet, but they really really should)
> 
> Right Fedora still uses a regular fstab entry for the ESP and changing
> that is out of scope for what I'm working on.

Well, but it might be worth supporting such setups anyway, no? i mean,
I am pretty sure it would be wise to not focus on legacy environments
when designing new stuff.

> > Hence, my recommendation would be: write a small service that is
> > pulled in by default.target, but orders itself after it. Then make
> > your changes from there. i.e. do it as final steps during boot, rather
> > than delay it to shutdown.
> 
> See above, currently this is for Fedora Workstation only, and the plan
> is actually to do set boot_success from the systemd user session
> (using pkexec with a user on console check to execute a new grub2-set-bootflag
> binarty which only supports changing a limited no env variables),
> so that we know that the user has actually logged in successfully
> before setting the flag.

Are you sure that powering up a system and powering it down
right-after should trigger the boot menu?

> > note that there have been plans of introducing some generic framework
> > for such "boot completion" tests, as it is useful for a number of
> > usecases, for example Atomic would like to use that. Such a framework
> > would be very minimal most likely: add a new generically named target,
> > before which all "is all good" checkers would be ordered, and after
> > which all "mark the boot as successful" servers ared ordered. Your
> > grub service would fit in perfectly in the latter then.
> 
> So something like this would make sense for server / container
> scenarios but not really for Fedora WS, if gdm starts but the
> keyboard is not functional so the user cannot log in we still
> want to show the menu the next boot so that the user can say
> try an older kernel.
> 
> And even in server / container scenarios ideally asserting
> success would come from a service which checks that say a http
> connection can be made or whatever depending on the role of
> the server...

Precisely, that's why the target unit would be pluggable: downstreams
can plug anything they like before it, so that the target would never
be reached if by any of the deps of the new target the system would
not be considered to be up. it's then up to downstream to define
servies and plug them in.

> ###
> 
> I cannot help but notice that you've not answered my original
> question. So given that the plan is to have both a boot_success
> and a shutdown_success flag how would you go about implementing
> the service for this?

Yes, because it's a bad idea.

Again, the are automounts for the ESP in place on many systems. We
generally refuse fulfilling automount requests during shutdown
however, so that we don't end up in endless cycles of
trigger-unmount-trigger-unmount and so on...

You can order your stuff After=local-fs.target and
Before=basic.target, but this is useless, and will fail for any system
with automounts for that in place. Seriously, don't waste your time on
this, it's pointless...

> As explained my current plan is a grub_shutdown.service which gets
> added to /lib/systemd/system/poweroff.target, etc. and has a:

No. To run code during shutdown, write a service that is started at
boot, and set its ExecStop= line (you can leave out ExecStart= as long
as there's ExecStop=) defined.

Neither poweroff.target, nor shutdown.target nor umount.target is
relevant to you. At that time these points are reached mounts will
already be gone. 

> Which I think you won't like, in which case I can live with just
> the Before=umount.target but I wonder what your thoughts are on
> this?

Please, this deserves some discussion before you implement this. And
shutdown hooks are really not the way to go.

Lennart

-- 
Lennart Poettering, Red Hat