On Sat, Apr 10, 2021 at 8:36 PM Chris Murphy <lists@xxxxxxxxxxxxxxxxx> wrote: > > I can reproduce the bolt testcase problem in a podman container, with > overlay driver, using ext4, xfs, and btrfs. So I think I can drop > linux-btrfs@ from this thread. > > Also I can reproduce the title of this thread simply by 'podman system > reset' and see the kernel messages before doing the actual reset. I > have a strace here of what it's doing: > > https://drive.google.com/file/d/1L9lEm5n4-d9qemgCq3ijqoBstM-PP1By/view?usp=sharing > I'm confused. The error in the title of the page is from overlayfs mount(). I see no mount in the strace. I feel that I am missing some info. Can you provide the overlayfs mount arguments and more information about the underlying layers? > It may be something intentional. The failing testcase, > :../tests/test-common.c:1413:test_io_dir_is_empty also has more > instances of this line, but I don't know they are related. So I'll > keep looking into that. > > > On Sat, Apr 10, 2021 at 2:04 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > > As the first step, can you try the suggested fix to ovl_dentry_version_inc() > > and/or adding the missing pr_debug() and including those prints in > > your report? > > I'll work with bolt upstream and try to further narrow down when it is > and isn't happening. > > > > I can reproduce this with 5.12.0-0.rc6.184.fc35.x86_64+debug and at > > > approximately the same time I see one, sometimes more, kernel > > > messages: > > > > > > [ 6295.379283] overlayfs: upper fs does not support xattr, falling > > > back to index=off and metacopy=off. > > > > > > > Can you say why there is no xattr support? > > I'm not sure. It could be podman specific or fuse-overlayfs related. Not sure how fuse-overlayfs is related. This is a message from overlayfs kernel driver. > Maybe something is using /tmp in one case and not another for some > reason? > > > Is the overlayfs mount executed without privileges to create trusted.* xattrs? > > The answer to that may be the key to understanding the bug. > > Yep. I think tmpfs supports xattr but not user xattr? And this example > is rootless podman, so it's all unprivileged. > OK, so unprivileged overlayfs mount support was added in v5.11 and it requires opt-in with mount option "userxattr", which could explain the problem if tmpfs is used as upper layer. Do you know if that is the case? I sounds to me like it may not be a kernel regression per-se, but a regression in the container runtime that started to use a new kernel feature? Need more context to understand. Perhaps the solution will be to add user xattr support to tmpfs.. Thanks, Amir.