On Wed, Sep 14, 2022 at 9:00 PM Hao Luo <haoluo@xxxxxxxxxx> wrote:
>
> On Tue, Sep 13, 2022 at 8:46 PM Amir Goldstein <amir73il@xxxxxxxxx> wrote:
> >
> > On Tue, Sep 13, 2022 at 11:33 PM Hao Luo <haoluo@xxxxxxxxxx> wrote:
> > >
> > > On Tue, Sep 13, 2022 at 11:54 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote:
> > > > OK. IIUC, you have upper fs files only in the root dir?
> > >
> > > Sorry, no, the upper fs files need to be in subdir.
> > >
> > > > And the lower root dir has only subdirs?
> > >
> > > There could be files.
> > >
> >
> > And assuming that those files are cgroupfs files, why
> > did you say there is no need to write to those files?
> >
> > I seem to recall that was an important distinction from
> > standard overlayfs when you described the problem in LSFMM.
>
> In my last reply, I was assuming all the writes to the cgroupfs files
> happen from remote, not from the union. One can read files from union,
> or create files that exist in the upper.
>
> The idea is, I can provide two copies of lower to users. One is the
> original lower, writable, so any update happens there. And the other
> is a union of the lower and the upper, it's a read-only view of the
> lower, but extended by the upper.
>
> I actually don't know whether supporting writes to the lower from the
> union is better. It probably is, because then I can combine the two
> copies into one.
>
Understood.
> >
> >
> > > > Can you give a small example of an upper a lower and their
> > > > union trees just for the sake of discussion?
> > > >
> > >
> > > For example, assume lower has the following layout:
> > > $ tree lower
> > > .
> > > └── A
> > > ├── B
> > > │ └── lower
> > > └── lower
> > >
> > > I can't create files in the fs in the lower.
> > > $ touch A/B/file
> > > touch: cannot touch 'A/B/file': Permission denied
> > >
> > > The upper is initially empty.
> > >
> > > I would like to overlay a writable fs on top of lower, so the union
> > > tree looks like
> > > $ tree union
> > > .
> > > └── A
> > > ├── B
> > > │ └── lower
> > > └── lower
> > > $ touch A/B/file
> > > $ tree union
> > > .
> > > └── A
> > > ├── B
> > > │ ├── file
> > > │ └── lower2
> > > └── lower1
> > >
> > > Here, 'file' exists in the upper.
> > >
> >
> > So B is now called a "merged" dir - it is not a "pure" dir
> > anymore because it contains both upper and lower files.
> >
> > Normally in overlayfs before creating 'file' in upper,
> > the hierarchy A/B/ needs to be created in upper fs
> > to contain the file.
> >
> > Unless your upper fs automagically has the same
> > dirs hierarchy as the lower fs?
> >
> > You should know that overlayfs does more than just
> > mkdir("A");mkdir("A/B")
> > it created tmp dirs, sets xattrs and attrs on them and moves
> > them into place.
> > I am not sure if you planned to support all those operations
> > in your upper fs?
> >
>
> Yeah. I can add support for tmp dirs, tmp files in my upper fs, that
> is, bpffs. I played it a bit back in May, that is totally doable. I
> remembered I successfully made bpffs accepted as overlayfs's upper
> without xattrs and attrs back then. Maybe I missed something.
>
sounds reasonable.
xattr is not a hard requirement but some things (that you do not need)
cannot work without xattr like merged directory rename and rmdir.
> > There are probably some other limitations at the moment
> > related to pseudo filesystems that prevent them from being
> > used as upper and/or lower fs in overlayfs.
> >
> > We will need to check what those limitations are and whether
> > those limitations could be lifted for your specific use case.
> >
>
> How can we approach this? Maybe I can send my patch that adds tmp dir,
> tmp files and xattr, attr to upstream as RFC, so you can take a look?
>
I don't think I need your fs to test.
The only thing special in this setup as far as I can tell is the dynamic
cgroupfs (or cgroup2?) lower dirs.
IIUC, everything worked for you except for oddities related to
lower directories not appearing and not disappearing from the union.
Is that correct? is that the only thing that you need a fix for?
> > > Further, directory B could disappear from lower. When that happens, I
> > > think there are two possible behaviors:
> > > - make 'file' disappear from union as well;
> > > - make 'file' and its directory accessible as well.
> > >
> > > In behavior 1, it will look like
> > > $ tree union
> > > .
> > > └── A
> > > └── lower1
> > >
> > > In behavior 2, it will look like
> > > $ tree union
> > > .
> > > └── A
> > > ├── B
> > > │ └── file
> > > └── lower1
> > >
> > > IMHO, behavior 1 works better in my use case. But if the FS experts
> > > think behavior 2 makes more sense, I can work around.
> > >
> >
> > Something that I always wanted to try is to get rid of the duplicated
> > upper fs hierarchy.
> >
> > It's a bit complicated to explain the details, but if your use case
> > does not involve any directory renames(?), then the upper path
> > for the merge directories can be index based and not hierarchical.
> >
>
> Yeah, I don't expect directory renaming. But I can't say if there is
> anyone trying to do that by accident, or by bad intention.
>
Your fs will return an error for rename if you did not implement it.
Anyway, if you can accept behavior 2, it is much more simple.
This other idea is very vague and not simple, so better not risk it.
If you confirm that you only need to get uptodate view of
lower dirs in union, then I will look for the patches that I have
and see if they can help you.
Thanks,
Amir.
