On Wed, Sep 2, 2020 at 4:29 PM Vivek Goyal <vgoyal@xxxxxxxxxx> wrote: > > On Wed, Sep 02, 2020 at 06:34:12AM +0300, Amir Goldstein wrote: > > On Wed, Sep 2, 2020 at 2:37 AM Mauro Condarelli <mc5686@xxxxxxxxx> wrote: > > > > > > Thanks Amir, > > > comments inline below > > > > > > Regards > > > Mauro > > > > > > On 9/1/20 8:43 PM, Amir Goldstein wrote: > > > > > > On Tue, Sep 1, 2020 at 9:01 PM Mauro Condarelli <mc5686@xxxxxxxxx> wrote: > > > > > > Hi, > > > most likely this is not the right place to ask, please redirect me as needed. > > > > > > I'm trying to use OverlayFS to add (limited) write capability to a ReadOnly > > > rootfs (SquashFS) > > > > > > Essentially (actual script is more complex, of course) boot-sequence includes: > > > > > > # /dev/mmcblk0p5: ext4 (upper+work+nwwroot+newroot/oldroot) > > > # /dev/mmcblk0p6: SquashFS mounted on / > > > mount /dev/mmcblk0p5 /overlay > > > mount -t overlay overlay -o lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work /overlay/newroot > > > cd /overlay/newroot > > > pivot_root . oldroot > > > mount --move oldroot/dev /dev > > > mount --move oldroot/proc /proc > > > > > > This works as expected, but, too often for comfort, some file > > > (and sometime also directories) become unavailable due to error: > > > > > > overlayfs: invalid origin (ssh/sshd_config, ftype=8000, origin ftype=4000). > > > > > > File name changes, of course, but rest is fairly constant. > > > > > > This always happens when some file is written. > > > Error persists reboots. > > > Only way I found to "cure" the system is to go on "upper" and delete the file > > > thus going back to "lower" version (in this case I should delete "/oldroot/overlay/upper/etc/ssh/sshd_config") > > > > > > This is a self-built kernel (Linux vocore 5.7.0 #2 PREEMPT Mon Aug 3 09:19:06 CEST 2020 mips GNU/Linux) > > > on a custom target based on a SoC (MT7628). > > > > > > I am available to do any required test, but I have no idea about where to start. > > > > > > Any hint (or redirect) would be greatly appreciated. > > > > > > This is probably your problem: > > > https://lore.kernel.org/linux-unionfs/32532923.JtPX5UtSzP@fgdesktop/ > > > > > > It surely looks like it is. > > > I tried to follow the thread, but I'm unsure i grokked it . > > > > > > I am using OverlayFS (ext4 over a SquashFS) on rootfs in order to > > > be able to update the whole system (changing SquashFS) while > > > retaining customization (essentially tweaks in /etc/, host certificates > > > and similar stuff) in an embedded target. > > > > > > To complicate matters I also have a dual system for fallback in case > > > of faulty upgrade; this means I can switch from: > > > lower=part6, upper=part5, update will go into part7 > > > to > > > lower=part7, upper=part5, update will go into part6 > > > > > > I don't need nfs at all, so exporting OverlayFS is not an issue, > > > but it's unclear to me if this "lower swapping" is > > > actually supported as I see: > > > > > > > > /me is again wondering what's the use case of modifying lower layer > > > > > with an existing upper. Is it fair to say, no don't recreate/modify > > > > > lower layers and use with existing upper. > > > > > > > > > > > > > It's fine by me to document that this is not supported. > > > > > > ... which is scary. > > > Note I do *not* need to modify lower on-the-fly, when I > > > swap systems, that will happen at reboot. > > > > Don't be scared :-) > > Your reaction is a good answer to Vivek's question above - > > No, it is not fair to say don't re-create lower and use existing upper > > This seems like a wrong use case to me. So in above example, say > following happens. > > - Mount overlay > - modify sshd_config (it gets copied up). > - Lower squahsfs gets updated and /etc/sshd/sshd_config gets updated. > - Mount overlay and user now sees old copied up sshd_config. (If it > works). > > Conceptually it is not making much sense to me. What's the point of > upgrading lower because after mounting overlay you might still see > old file. And to make it worse, behavior is underministic beacuse > if file has not been copied up, you will see new file. > That is exactly the case with package managers that update system files, except package managers can prompt user to resolve conflicts. Think of this is a package manager that manages a single package called "system". If that package manager (usually called firmware upgrade) wanted to, it could compare for every upper file to origin from system image A and to same path from system image B before making the upgrade and prompt the user to resolve the conflict just like a regular package manager. But if said package manager does not resolve conflicts it is the same as a package manager configured to auto resolve conflicts by choosing the local modification. Not perfect, but people do live with such setups even without overlayfs. Thanks, Amir.
