On Tue, Dec 7, 2021 at 11:17 PM Carl Karsten <carl@xxxxxxxxxxxxxxxx> wrote:
>
> On Mon, Dec 6, 2021 at 9:55 PM Amir Goldstein <amir73il@xxxxxxxxx> wrote:
> >
> > On Tue, Dec 7, 2021 at 4:38 AM Carl Karsten <carl@xxxxxxxxxxxxxxxx> wrote:
> > >
> > > I powercyced my box, I'm assuming I did a sudo poweroff or halt or
> > > something, but maybe not.
> > >
> > > It got stuck, (enter rescue mode, root account disabled, hit enter to
> > > enter rescue mode... ) so I killed power.
> > >
> > > I edited fstab, changed auto to noauto, booted.
> > >
> > > overlay /srv/nfs/rpi/bullseye/boot/merged overlay
> > > noauto,defaults,ro,lowerdir=/srv/nfs/rpi/bullseye/boot/updates:/srv/nfs/rpi/bullseye/boot/setup:/srv/nfs/rpi/bullseye/boot/base,upperdir=/srv/nfs/rpi/bullseye/boot/play,workdir=/srv/nfs/rpi/bullseye/boot/work,nfs_export=on
> > > 0 0
> > >
> > > # mount /srv/nfs/rpi/bullseye/boot/merged
> > > mount: /srv/nfs/rpi/bullseye/boot/merged: mount(2) system call failed:
> > > Stale file handle.
> > >
> > > dmesg shows:
> > >
> > > [ 45.941350] overlayfs: failed to verify upper (boot/play,
> > > ino=379405, err=-116)
> > > [ 45.941369] overlayfs: failed to verify index dir 'upper' xattr
> > > [ 45.941379] overlayfs: try deleting index dir or mounting with '-o
> > > index=off' to disable inodes index.
> > >
> >
> > Did you by any change re-create boot/play dir without re-creating
> > boot/work dir? because that is what that message means.
> > I doubt it has to do with the power cycle.
>
> Maybe.
>
> would that include:
> mount: lower=base,setup upper= updates
> unmount
> mount: lower=base,setup,updates upper=play
> ?
>
> That might have happened.
Yes, that would do it.
>
> >
> > > It is a bit concerning that I need to consider a power cycle may
> > > require local console use. I plan on managing this remotely. For now
> > > I'll leave it noauto and remotely mount when needed. so this isn't a
> > > critical blocker to me.
> > >
> > > What would happen: errors=remount-ro
> > >
> > > errors={continue|remount-ro|panic}
> > > Define the behavior when an error is encountered. The
> > > default is set in the filesystem superblock, ... and can be
> > > changed using tune2fs(8).
> > >
> >
> > This looks like a quote from the man page section about ext2 mount options.
> > Those are not options supported for overlayfs.
> >
> > Anyway, with that error, as the kernel log says, you could have mounted
> > overlayfs ro with -o index=off and we could have added a mount option
> > to fallback to ro mount with index=off, but that means no nfs_export,
> > so not sure if that is what you wanted.
>
> That would be an improvement, as would errors=continue.
>
> Either of these would let a healthy system come up and be usable, and
> errors would let the system come up (finish booting) but the
> nfs/overlay part needs attention.
>
> Currently I have noauto so any reboot requires attention.
> If I set auto, an error will cause the boot to stop and I don't have a
> way to fix it remotely. I don't want to risk that, so noauto.
>
There is no need for a kernel change.
You can use boot scripts or mount helper to implement the mount options
fallback that suits your system needs.
> I think I would prefer errors=continue as It would be more obvious
> that the overlayfs mount has issues.
>
That is not likely to happen and FYI errors=continue does not even
mean what you think for ext4 where it is supported.
It does not continue on failure to mount, it continues on IO and
fs corruption errors detected during operations on a mounted fs.
Thanks,
Amir.
