ah, non-disk filesystem is the problem. thanks. I had switched the mounts to use ..../base and everything was fine, so I figured overlayfs was mucking something up. I'm glad you enjoying my riddles :) On Thu, Oct 21, 2021 at 2:09 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > On Thu, Oct 21, 2021 at 12:51 AM Carl Karsten <carl@xxxxxxxxxxxxxxxx> wrote: > > > > I have 2 overlayfs monts. > > > > I can nfs export one or the other fine, but not both at the same time, > > the client gets the wrong files. > > Hi Carl, > > Thanks for the daily overlayfs riddles ;-) > > This is not an overlayfs issue. It's nfs exports configuration issue. > Please read the documentation of fsid export option. > Disk filesystems have uuid so they don't need explicit identification. > Overlayfs does not have a stable UUID. > > > > > server hosthame negk bullseye ext4 > > client twist ubuntu > > > > juser@negk:~$ cat /etc/exports > > # /srv/nfs/rpi/buster/boot/merged *(ro,sync,no_subtree_check,no_root_squash) > > /srv/nfs/rpi/buster/root/merged *(ro,sync,no_subtree_check,no_root_squash) > > In this case, "root" is auto assigned fsid 1 > > > > > juser@negk:~$ ls /srv/nfs/rpi/buster/boot/merged > > bcm2708-rpi-b.dtb bcm2710-rpi-3-b-plus.dtb fixup4cd.dat > > kernel7.img start4.elf > > (snip) > > > > juser@negk:~$ ls /srv/nfs/rpi/buster/root/merged > > bin boot dev etc home lib lost+found media mnt opt proc > > root run sbin srv sys tmp usr var > > > > carl@twist:~/mnt$ sudo mount -t nfs negk:/srv/nfs/rpi/buster/root/merged nfs > > carl@twist:~/mnt$ ls nfs > > bin boot dev etc home lib lost+found media mnt opt proc > > root run sbin srv sys tmp usr var > > carl@twist:~/mnt$ sudo umount nfs > > > > # enable root in negk exports... > > I think you meant enable "boot" > and is that case "boot" gets assigned fsid 1 and "root" gets reassigned fsid 2 > > > > > carl@twist:~/mnt$ sudo mount -t nfs negk:/srv/nfs/rpi/buster/root/merged nfs > > carl@twist:~/mnt$ ls nfs > > bcm2708-rpi-b.dtb bcm2710-rpi-3-b-plus.dtb fixup4cd.dat > > kernel7.img start4.elf > > (snip) > > > > It is a bit surprising to me that you get the wrong export after a clean mount, > but it doesn't change the core configuration issue. > > Perhaps nfs client has some fsid cache or you have a deferred umount in > the system of negk:/srv/nfs/rpi/buster/root/merged (e.g. due to spawned mount > ns or something) and then the "new" nfs mount uses the old resolved fsid 1. > > In any case, when exporting more than one non-disk filesystem, you should > use explicit fsid. > > Thanks, > Amir. -- Carl K
