Dealing with read-only rootfs

[lennart@xxxxxxxxxxxxxx (Lennart Poettering)]

On So, 17.06.18 18:58, Jérémy Rosen (jeremy.rosen at smile.fr) wrote:

> Hello everybody
> 
> I am trying to understand the recommanded way to deal with read-only
> rootfs...

Have you seen this btw:

http://0pointer.net/blog/projects/stateless.html

It's from 2014, and a number of things happened since them, but it
introduces the basic concepts.

> my understanding is that (slightly simplified)
> * /run must be a tmpfs

Well, this is the case for all systems, not just those with read-only
root. systemd will mounte /run as tmpfs for you anyway and in any
case, it's not something you ever have to think about. It's basically
in the same category as /sys and /dev: it's mounted implicitly and not
up to distributors or even users to change. Or to say this
differently: if you want to use something else than tmpfs on /run then
you are welcome, but you are on your own, and systemd won't make that
easy for you.

> * /var must be writable, and will usually be persistant (can be a
>   tmpfs too)

Correct. Also see file-hierarchy(7).

> * everything else can be read-only

Yes, in particular /etc and /usr can be read-only.

> My problem comes with populating an empty /var. There seems to be hints to a
> systemd-provided mechanism to do that, but I can't find any documentation
> for it

systemd will set everything up as needed automatically for you, and
well-behaved packages do that too. There are generally three options
to make sure that a package has everything it needs under /var:

1. Add tmpfiles.d/ snippets to create the dirs that are needed if they
   are missing. systemd ships some basic ones, that set up the basic
   hierarchy, and everything it needs for its own components. But 3rd
   party package can install whatever they want on top. Note that
   tmpfiles.d/ knows the "C" line which copies whole directory trees
   into /var if that is needed. It defaults to copying those from
   /usr/share/factory, but you actually can copy them from anywhere
   you like. Usually using "C" for /var is not really necessary
   though, as most software is happy with just a few properly owned
   dirs bein created for them, and doesn#t need any copied fields. "C"
   is very handy however if you also start with an empty /etc, because
   it can then be used to do some minimal population of /etc, which is
   necessary for more software to work correctly.

2. Add StateDirectory=, CacheDirectory=, LogsDirectory= to your
   service file. Many services only need a directory or two underneath
   /var/lib/, /var/cache/ and /var/log/, and if that's the case it's
   much nicer to use these three settings to embedd this info directly
   in the unit file.

3. Fix the service in question to create evertyhing it needs
   implicitly on first run. This of course is the most robust
   solution, but there are limits to this, for example if the service
   never possesses privileges.

> Apparently, tmpfiles.d should copy the content from /usr/share/factory to
> /var when /var is empty, but I can't find what actually implements that
> 
> * is it dynamically added in /run ?
> * is it just a recommanded way, and I have to implemented myself ? if yes,
> is the directory /usr/share/factory documented anywhere ?

Yes, in both file-hierarchy(7) and tmpfiles.d(5).

> * what qualifies as "empty /var" ? (this is mostly about being robust to
> power-loss during the copying of /var)

tmpfiles snippets run on every boot and adjust what is necessary but
are happy to use what is already there. This means that they can
safely run on every boot, and will continue where a previous run left
of if necessary.

Lennart

-- 
Lennart Poettering, Red Hat