On Fri, Dec 01, 2023 at 04:02:50PM -0600, Rob Landley wrote:
You are reasoning backwards from your solution and not thinking about
the
design. I don't think you're addressing the real issue.
Right now "separate" container namespaces all share a common rootfs instance.
They do NOT share a common init task, even though before containers that was
universal. You can have your own PID namespace, which starts _empty_.
Your mount tree in a container does NOT start empty. From the clone(2) man page:
If CLONE_NEWNS is set, the cloned child is started in a new
mount namespace, initialized with a copy of the namespace of the parent.
Defaulting to having everything in it and removing what you don't want to keep
is very different from what PID or UID namespaces do, and is causing you
problems. Doing a chroot is basically an overmount, the other mount points are
still there in your tree and accessable if you try hard enough, and rootfs is
common to all containers. Mitigating this requires cleanup work that isn't
always even possible to fully do (ala rootfs actually being used, which does
happen a lot today and it's always accessible if a static process forking its
own mount namespace does enough umounts, which can then act as a
cifs/nfs/9p/rsync server out to the parent or some such).
Logically, extending the kernel to have a CLONE_NEWROOTFS where it gets a _new_
ramfs or tmpfs instance, unique to that namespace, at the root of a new empty
mount tree, is the logical fix. There is then design work around "so what API do
you use to populate it" which could range from "the first int below child_stack
is the fd of a cpio.gz to extract into it and then it launches an /init out of
there the way the host linux boots" through "the new child starts suspended ala
vfork/ptrace and then the parent process initializes it and unblocks it" to "the
init task is running the executable from the host context that called clone and
has inherited the existing open filehandles from the host context, although
despite the openat() family being in posix-2008 we sadly don't appear to have a
mountat()...". I dunno. That's design work to properly fix the issue.
You don't want to address the design problem, you want to add a special case
workaround for your current issue. You see doing that as a "design fix". I do not.
I think this is a good point - I definitely agree that the weird
hackiness that runtimes have to do to setup their mount namespaces
properly is suboptimal.
The hypothetical CLONE_NEWROOTFS that you suggest is a superior
suggestion - not least because it would better do what containers
actually want, but it would also do it with less syscalls and flapping!
As an aside: I take your point RE rootfs being shared. The general
concern is normally that information from the host might leak if
containers can read the host root, so sharing an empty rootfs is less of
a concern, but again the theoretical case of information sharing between
containers by writing to the shared rootfs is an interesting one too.
Fine. Moving on. I still think a dedicated CONFIG entry is a bad way to do the
silly thing. Specifying the silly thing on the kernel command line seems less bad.
Checking for "root=tmpfs" to trigger the silly thing seems less bad to
me,
although I note that init/do_mounts.c function init_rootfs() already _is_
checking for that (and there's a pending patch to tweak it), so... be aware.
My original reasoning for having it as a built option was that, in the
case of running directly from initramfs, that's often something that's
done if you're embedding the initfamfs to create a unified kernel. As a
result, it is something that you'd only really care to turn on or off at
build time.
Having said that, I have no strong opinion on that.
That's the part I don't understand. It _seems_ like what you were
saying. Not
"this hasn't been working fine for everyone else for the past 15 years already",
but "I think it should have been designed a different way 20 years ago, and
would like to change it to match my opinion".
I have to say I struggle to understand where to go from here... as I
said above, I do like the CLONE_NEWROOTFS suggestion (and it was
actually something I was batting around for my own project) but that
feels that a _way more_ specialised feature.
And now you are saying that apparently we _shouldn't_ make a relatively
small change to initramfs because its worked fine for years, but we
should add a much larger patch to clone() which has also worked for many
years? I shouldn't question how initramfs works because you were there
when it was written [1], but we should question all the devs who decided
on CLONE_NEWNS over CLONE_NEWROOTFS?
I'm not saying we shouldn't, but help me out here - how can I tell
what's "reasonable" to question and what isn't?
[1]: https://media.tenor.com/lR9rjwXjL50AAAAC/deep-magic-lion.gif
LOTS of embedded people have used the existing initramfs, and it's accumulated a
BUNCH of weirdness over the years. Did you know you can concatenate multiple
cpio.gz files and the kernel loader will accept them as one big
archive?
I did, yes.
Are you suggesting I don't understand because I'm not "one of us"?
No, and I am sorry that I phrased that poorly. I merely meant that there
are a hell of a lot of different build options and systems within the
kernel, and it is perhaps not unreasonable to suggest that it is not a
requirement that everyone intimately understands all of them all of the
time.
You are not the first person to use this plumbing. "Everybody _really_
wants
what I think it should always have been like, but nobody's mentioned it in the
past 20 years" is a strange position to take. Earlier you said "the fact that
the desirable path is" as a universal statement rather than a personal opinion.
Desirable to who? Judged as "fact" by who?
I meant for container runtimes. Most are quite opinionated about not
doing mount --move . / && chroot(.), strictly preferring pivot_root
instead.
--
Emily Shepherd
Red Coat Development Limited
