On Tue, Sep 13, 2022 at 9:26 PM Hao Luo <haoluo@xxxxxxxxxx> wrote: > > On Mon, Sep 12, 2022 at 11:28 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > > > On Mon, Sep 12, 2022 at 8:43 PM Hao Luo <haoluo@xxxxxxxxxx> wrote: > > > > > > On Mon, Sep 12, 2022 at 8:40 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > > > > > > > On Mon, Sep 12, 2022 at 5:22 PM Miklos Szeredi <miklos@xxxxxxxxxx> wrote: > > > > > > > > > > I imagine the "switch" layer for a HSM would be simple enough: > > > > > > > > > > a) if file exists on fastest layer (upper) then take that > > > > > b) if not then fall back to fuse layer (lower) . > > > > > > > > > > It's almost like a read-only overlayfs (no copy up) except it would be > > > > > read-write and copy-up/down would be performed by the server as > > > > > needed. No page cache duplication for upper, and AFAICS no corner > > > > > cases that overlayfs has, since all layers are consistent (the fuse > > > > > layer would reference the upper if that is currently the up-to-date > > > > > one). > > > > > > > > On recent LSF/MM/BPF, BPF developers asked me about using overlayfs > > > > for something that looks like the above - merging of non overlapping layers > > > > without any copy-up/down, but with write to lower. > > > > > > > > I gave them the same solution (overlayfs without copy-up) > > > > but I said I didn't know what you would think about this overlayfs mode > > > > and I also pointed them to the eBPF-FUSE developers as another > > > > possible solution to their use case. > > > > > > > > > > Thanks Amir for adding me in the thread. This idea is very useful for > > > BPF use cases. > > > [...] > > > > Am I to understand that the eBPF-FUSE option currently > > does not fit your needs (maybe because it is not ready yet)? > > > > Yeah, mostly because eBPF-FUSE is not ready. I talked to Alessio and > his colleague after LSF/MM/BPF. They were distracted from eBPF-FUSE > development at that time and I didn't follow up, working on other BPF > stuff. > > > > > [...] > > Can't say I was able to understand the description, but let me > > try to write the requirement in overlayfs terminology. > > Please correct me if I am wrong. > > > > 1. The "lower" fs (cgroupfs?) is a "remote" fs where directories > > may appear or disappear due to "remote" events > > > > Right. Seems we are aligned on this. > > > I think there were similar requirements to support changes > > to lower fs which in a network fs in the past. > > As long as those directories are strictly lower that should be > > possible. > > > > Does upper fs have directories of the same name that need to > > be merged with those lower dirs? > > > > No, I don't think so. Upper fs should only have files in my use case. > > > 2. You did not mention this but IIRC, the lower fs has (pseudo) > > files that you want to be able to write to, and those files never > > exist in the upper fs > > > > That translates to allowing write to lower fs without triggering copy-up > > > > Writing to lower is not needed right now. > > > 3. The upper fs files are all "pure" - that means those file paths > > do not exist in lower fs > > > > Creating files in the upper fs is normal in overlayfs, so not a problem. > > > > Yes. If used in an expected way, the lower fs won't have file paths > that also exist in the upper. > > > 4. Merging upper and lower directory content is usually not needed?? > > > > It must be needed for the root dir at least but where else? > > In which directories are the upper files created? > > Which directories are "pure" (containing either lower or upper files) > > and which directories are "merge" of upper and lower children? > > > > In my use case, if that's doable, all the directories are "pure", > except the root dir, and they are in the lower. The files are either > from upper or from lower, so no merge. This should be sufficient for > the BPF use case. > > > If the answer is that some directories exist both in upper and in lower > > and that those directories should be merged for the unified view then > > this is standard overlayfs behavior. > > > > Once I get the requirement in order I will try to write coherent > > semantics for this mode and see if it can meet the needs of other > > projects, such as HSM with FUSE as lower fs. > > Thanks Amir. I want to clarify that with my very very limited > knowledge on FS, I can't say what I am thinking right now is > absolutely correct. Please take my request for features with a grain > of salt and feel free to pick the semantics that you see make most > sense. > OK. IIUC, you have upper fs files only in the root dir? And the lower root dir has only subdirs? If that is all then it sounds pretty simple. It could be described something like this: 1. merged directories cannot appear/disappear 2. lower pure directories can appear/disappear 3. upper files/dirs can be created inside merge dirs and pure upper dirs I think I have some patches that could help with #2. Can you give a small example of an upper a lower and their union trees just for the sake of discussion? Thanks, Amir.
