On Mon, Aug 21, 2023 at 3:26 PM Amir Goldstein <amir73il@xxxxxxxxx> wrote:
>
> On Mon, Aug 21, 2023 at 3:55 PM Alexander Larsson <alexl@xxxxxxxxxx> wrote:
> >
> > On Mon, Aug 21, 2023 at 1:00 PM Miklos Szeredi <miklos@xxxxxxxxxx> wrote:
> > >
> > > On Thu, 17 Aug 2023 at 13:05, Alexander Larsson <alexl@xxxxxxxxxx> wrote:
> > > >
> > > > This is needed to properly stack overlay filesystems, I.E, being able
> > > > to create a whiteout file on an overlay mount and then use that as
> > > > part of the lowerdir in another overlay mount.
> > > >
> > > > The way this works is that we create a regular whiteout, but set the
> > > > `overlay.nowhiteout` xattr on it. Whenever we check if a file is a
> > > > whiteout we check this xattr and don't treat it as a whiteout if it is
> > > > set. The xattr itself is then stripped and when viewed as part of the
> > > > overlayfs mount it looks like a regular whiteout.
> > > >
> > >
> > > I understand the motivation, but don't have good feelings about the
> > > implementation. Like the xattr escaping this should also have the
> > > property that when fed to an old kernel version, it shouldn't
> > > interpret this object as a whiteout. Whether it remains hidden like
> > > the escaped xattrs or if it shows up as something else is
> > > uninteresting.
> > >
> > > It could just be a zero sized regular file with "overlay.whiteout".
> > >
> > > But we are also getting to the stage where the number of getxattr
> > > queries on lookup could be a performance problem. Or maybe not. It
> > > would be good to look at this aspect as well when adding xattr queries
> > > to lookup.
> >
> > Wanting to avoid (as much as possible) the reading of more xattrs
> > which would affect performance of every regular file was the reason
> > for this particular implementation. I will do some more thinking and
> > see if I can come up with an alternative approach.
>
> I'd just like to add that, although the cost of getxattr is mostly fs
> dependent, from my experience, the cost of several getxattr on one
> file are usually amortized in the cost of the first getxattr on that file.
>
> So while it is valuable to avoid any getxattr if possible, avoid an
> extra getxattr is often less critical.
> Again, this statement may not be accurate for all fs.
I dunno. Each time I look at the xattr codepaths they seem very long,
and there is no vfs-side caching. Each getxattr() will invoke the
entire stack of reading and (re)parsing the metadata blocks needed for
xattr info. However, cpus are fast, and typically the blocks
themselves are cached in the buffer cache, so maybe this doesn't
necessarily matter.
To check this I ran some quick performance tests. A single, cached
(i.e. repeated), missed (e.g. ENODATA) getxattr call on xfs and ext4
takes about 1-1.5 microseconds. This is about the same timescale as a
stat call, so pretty fast I guess (and probably even faster when the
kernel does it, as there is no syscall overhead). However, on e.g. a
fuse mount (I used the passthrough_ll example) it takes about 20 usec
because each call must be roundtripped via the fuse daemon. Similarly
I can imagine a network filesystem may be slow here, although I see
that NFS4 has an xattr cache.
So, I guess the end result is that it's probably ok to use an extra
getxattr here, and that fuse should probably grow an xattr cache.
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Alexander Larsson Red Hat, Inc
alexl@xxxxxxxxxx alexander.larsson@xxxxxxxxx
