Hi Vivek, comments inline below. On 9/2/20 3:29 PM, Vivek Goyal wrote: > On Wed, Sep 02, 2020 at 06:34:12AM +0300, Amir Goldstein wrote: >> On Wed, Sep 2, 2020 at 2:37 AM Mauro Condarelli <mc5686@xxxxxxxxx> wrote: >>> Thanks Amir, >>> comments inline below >>> >>> Regards >>> Mauro >>> >>> On 9/1/20 8:43 PM, Amir Goldstein wrote: >>> >>> On Tue, Sep 1, 2020 at 9:01 PM Mauro Condarelli <mc5686@xxxxxxxxx> wrote: >>> >>> Hi, >>> most likely this is not the right place to ask, please redirect me as needed. >>> >>> I'm trying to use OverlayFS to add (limited) write capability to a ReadOnly >>> rootfs (SquashFS) >>> >>> Essentially (actual script is more complex, of course) boot-sequence includes: >>> >>> # /dev/mmcblk0p5: ext4 (upper+work+nwwroot+newroot/oldroot) >>> # /dev/mmcblk0p6: SquashFS mounted on / >>> mount /dev/mmcblk0p5 /overlay >>> mount -t overlay overlay -o lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work /overlay/newroot >>> cd /overlay/newroot >>> pivot_root . oldroot >>> mount --move oldroot/dev /dev >>> mount --move oldroot/proc /proc >>> >>> This works as expected, but, too often for comfort, some file >>> (and sometime also directories) become unavailable due to error: >>> >>> overlayfs: invalid origin (ssh/sshd_config, ftype=8000, origin ftype=4000). >>> >>> File name changes, of course, but rest is fairly constant. >>> >>> This always happens when some file is written. >>> Error persists reboots. >>> Only way I found to "cure" the system is to go on "upper" and delete the file >>> thus going back to "lower" version (in this case I should delete "/oldroot/overlay/upper/etc/ssh/sshd_config") >>> >>> This is a self-built kernel (Linux vocore 5.7.0 #2 PREEMPT Mon Aug 3 09:19:06 CEST 2020 mips GNU/Linux) >>> on a custom target based on a SoC (MT7628). >>> >>> I am available to do any required test, but I have no idea about where to start. >>> >>> Any hint (or redirect) would be greatly appreciated. >>> >>> This is probably your problem: >>> https://lore.kernel.org/linux-unionfs/32532923.JtPX5UtSzP@fgdesktop/ >>> >>> It surely looks like it is. >>> I tried to follow the thread, but I'm unsure i grokked it . >>> >>> I am using OverlayFS (ext4 over a SquashFS) on rootfs in order to >>> be able to update the whole system (changing SquashFS) while >>> retaining customization (essentially tweaks in /etc/, host certificates >>> and similar stuff) in an embedded target. >>> >>> To complicate matters I also have a dual system for fallback in case >>> of faulty upgrade; this means I can switch from: >>> lower=part6, upper=part5, update will go into part7 >>> to >>> lower=part7, upper=part5, update will go into part6 >>> >>> I don't need nfs at all, so exporting OverlayFS is not an issue, >>> but it's unclear to me if this "lower swapping" is >>> actually supported as I see: >>> >>>>> /me is again wondering what's the use case of modifying lower layer >>>>> with an existing upper. Is it fair to say, no don't recreate/modify >>>>> lower layers and use with existing upper. >>>>> >>>> It's fine by me to document that this is not supported. >>> ... which is scary. >>> Note I do *not* need to modify lower on-the-fly, when I >>> swap systems, that will happen at reboot. >> Don't be scared :-) >> Your reaction is a good answer to Vivek's question above - >> No, it is not fair to say don't re-create lower and use existing upper > This seems like a wrong use case to me. So in above example, say > following happens. > > - Mount overlay > - modify sshd_config (it gets copied up). > - Lower squahsfs gets updated and /etc/sshd/sshd_config gets updated. > - Mount overlay and user now sees old copied up sshd_config. (If it > works). > > Conceptually it is not making much sense to me. What's the point of > upgrading lower because after mounting overlay you might still see > old file. And to make it worse, behavior is underministic beacuse > if file has not been copied up, you will see new file. It might not make much sense to you, but it's EXACTLY what I need. This OverlayFS is used *only* for permanent configuration. Most of the times I'm adding new files (like host certificates and similar things), but sometimes applications provide "default configuration" I might need to override. If/when applications "upgrade their defaults" I definitely do not want them to override my choices. > To me, this sounds like a volatile overlay use case. Any changes to > overlay should be thrown away when lower sqashfs is updated and a > fresh upper should be set. I beg to differ. While You are right in the general case it's possible to end up with a tangled mess of old/new files none can sort out, your solution resembles a lot to Alexander "solution" of the gordian knot. In practice problems can (and will) arise in development when most of changes are done by hand for testing (so some unwanted "permanent" change can creep in; in production there's a well defined flow that should prevent mishap. Some of those configurations could be handled via other means, but they are invariably more complex and less adaptable to "unforeseen" needs/changes (which *will* happen). > Thanks > Vivek Regards Mauro
