On Tue, Sep 13, 2022 at 8:46 PM Amir Goldstein <amir73il@xxxxxxxxx> wrote:
>
> On Tue, Sep 13, 2022 at 11:33 PM Hao Luo <haoluo@xxxxxxxxxx> wrote:
> >
> > On Tue, Sep 13, 2022 at 11:54 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote:
> > > OK. IIUC, you have upper fs files only in the root dir?
> >
> > Sorry, no, the upper fs files need to be in subdir.
> >
> > > And the lower root dir has only subdirs?
> >
> > There could be files.
> >
>
> And assuming that those files are cgroupfs files, why
> did you say there is no need to write to those files?
>
> I seem to recall that was an important distinction from
> standard overlayfs when you described the problem in LSFMM.
In my last reply, I was assuming all the writes to the cgroupfs files
happen from remote, not from the union. One can read files from union,
or create files that exist in the upper.
The idea is, I can provide two copies of lower to users. One is the
original lower, writable, so any update happens there. And the other
is a union of the lower and the upper, it's a read-only view of the
lower, but extended by the upper.
I actually don't know whether supporting writes to the lower from the
union is better. It probably is, because then I can combine the two
copies into one.
>
>
> > > Can you give a small example of an upper a lower and their
> > > union trees just for the sake of discussion?
> > >
> >
> > For example, assume lower has the following layout:
> > $ tree lower
> > .
> > └── A
> > ├── B
> > │ └── lower
> > └── lower
> >
> > I can't create files in the fs in the lower.
> > $ touch A/B/file
> > touch: cannot touch 'A/B/file': Permission denied
> >
> > The upper is initially empty.
> >
> > I would like to overlay a writable fs on top of lower, so the union
> > tree looks like
> > $ tree union
> > .
> > └── A
> > ├── B
> > │ └── lower
> > └── lower
> > $ touch A/B/file
> > $ tree union
> > .
> > └── A
> > ├── B
> > │ ├── file
> > │ └── lower2
> > └── lower1
> >
> > Here, 'file' exists in the upper.
> >
>
> So B is now called a "merged" dir - it is not a "pure" dir
> anymore because it contains both upper and lower files.
>
> Normally in overlayfs before creating 'file' in upper,
> the hierarchy A/B/ needs to be created in upper fs
> to contain the file.
>
> Unless your upper fs automagically has the same
> dirs hierarchy as the lower fs?
>
> You should know that overlayfs does more than just
> mkdir("A");mkdir("A/B")
> it created tmp dirs, sets xattrs and attrs on them and moves
> them into place.
> I am not sure if you planned to support all those operations
> in your upper fs?
>
Yeah. I can add support for tmp dirs, tmp files in my upper fs, that
is, bpffs. I played it a bit back in May, that is totally doable. I
remembered I successfully made bpffs accepted as overlayfs's upper
without xattrs and attrs back then. Maybe I missed something.
> There are probably some other limitations at the moment
> related to pseudo filesystems that prevent them from being
> used as upper and/or lower fs in overlayfs.
>
> We will need to check what those limitations are and whether
> those limitations could be lifted for your specific use case.
>
How can we approach this? Maybe I can send my patch that adds tmp dir,
tmp files and xattr, attr to upstream as RFC, so you can take a look?
> > Further, directory B could disappear from lower. When that happens, I
> > think there are two possible behaviors:
> > - make 'file' disappear from union as well;
> > - make 'file' and its directory accessible as well.
> >
> > In behavior 1, it will look like
> > $ tree union
> > .
> > └── A
> > └── lower1
> >
> > In behavior 2, it will look like
> > $ tree union
> > .
> > └── A
> > ├── B
> > │ └── file
> > └── lower1
> >
> > IMHO, behavior 1 works better in my use case. But if the FS experts
> > think behavior 2 makes more sense, I can work around.
> >
>
> Something that I always wanted to try is to get rid of the duplicated
> upper fs hierarchy.
>
> It's a bit complicated to explain the details, but if your use case
> does not involve any directory renames(?), then the upper path
> for the merge directories can be index based and not hierarchical.
>
Yeah, I don't expect directory renaming. But I can't say if there is
anyone trying to do that by accident, or by bad intention.
> IOW:
> - union tree lookup/traversal by path is performed on the lower fs
> - upper fs is traversed by name only for pure upper dirs
> - merged dirs are found by index of lower dir inode
>
> The result will be behavior 1 that you wanted
>
> I have some other use cases that might benefit from this mode.
>
That will be very cool!
> > >
> > > If that is all then it sounds pretty simple.
> > > It could be described something like this:
> > > 1. merged directories cannot appear/disappear
> > > 2. lower pure directories can appear/disappear
> > > 3. upper files/dirs can be created inside merge dirs and pure upper dirs
> > >
> > > I think I have some patches that could help with #2.
> > >
> >
> > These three semantics looks good to me.
> >
>
> Except the case about disappearing B that you just described
> above breaks rule #1 ;)
> so other semantics are needed.
>
Yes :) Now I understand. Thanks for the explanation on "merged" dir.
> I will need some more clarifications about your use case to
> understand if what I have in mind could help your use case.
>
> Thanks,
> Amir.
