On Wed, Jan 31, 2018 at 9:05 PM, Vivek Goyal <vgoyal@xxxxxxxxxx> wrote: > On Wed, Jan 31, 2018 at 07:08:18PM +0100, Miklos Szeredi wrote: >> On Wed, Jan 31, 2018 at 5:55 PM, Vivek Goyal <vgoyal@xxxxxxxxxx> wrote: >> > On Wed, Jan 31, 2018 at 05:10:28PM +0100, Miklos Szeredi wrote: >> >> On Wed, Jan 31, 2018 at 4:58 PM, Amir Goldstein <amir73il@xxxxxxxxx> wrote: >> >> > On Wed, Jan 31, 2018 at 5:46 PM, Vivek Goyal <vgoyal@xxxxxxxxxx> wrote: >> >> >> On Wed, Jan 31, 2018 at 05:38:45PM +0200, Amir Goldstein wrote: >> >> >>> On Wed, Jan 31, 2018 at 5:20 PM, Vivek Goyal <vgoyal@xxxxxxxxxx> wrote: >> >> >>> > On Wed, Jan 31, 2018 at 03:57:12PM +0200, Amir Goldstein wrote: >> >> >>> > >> >> >>> > ORIGIN behavior is little unintuitive though. Despite the fact that file >> >> >>> > is not searchable through lower, it is visible through decoding of file >> >> >>> > handle and it is atleast non-intuitive. >> >> >>> > >> >> >>> >> >> >>> Maybe not intuitive at first glance, but try again: >> >> >>> >> >> >>> The only thing we *need* from underlying fs is to provide us with a unique >> >> >>> and persistent inode number we can use for the overlay object. >> >> >>> >> >> >>> Even if the inode number we get from underlying fs is not in any of the >> >> >>> layers, it is still a viable inode number we can use in overlay coupled >> >> >>> with overlay unique st_dev, to create a system wide unique st_dev;st_ino >> >> >>> tuple. >> >> >> >> >> >> As long as we use only inode number, it probably is still fine. >> >> >> >> >> >> But I look at ORIGIN as a generic infrastructure which other features can >> >> >> make use of it. For example, metacopy is using it to copy up file later. >> >> >> And there it will be non-intuitive that a file is not in any of the >> >> >> lower, still ORIGIN was decoded and file was copied up. It can come >> >> >> as a surprise to user. Atleast I was surprised when I ran into this >> >> >> while testing the feature. >> >> >> >> How about using REDIRECT for metacopy origin? Keeping ORIGIN only >> >> for inode, also meaning ORIGIN is only ever used on upper layer, never >> >> on middle layers. >> > >> > Hi Miklos, >> > >> > Trying to understand it better. So proposal seems to be that when a file >> > is copied up metacopy only, we store both REDIRECT and ORIGIN in upper >> > inode. When traversing metacopy inode chain, use ORIGIN info on upper >> > inode and REDIRECT info on lower/midlayer metacopy inode. >> > >> > I am assuming that this is to handle the use case of tar of upper layer >> > and untaring it as lower layer. >> > >> > One of the concerns Amir had raised with usage of REDIRECT was that it >> > will be significantly slower as comapred to decoding ORIGIN. So by using >> > ORIGIN on upper, we are trying to mitigate it up to some extent? We will >> > still pay the cost of decoding REDIECT in midlayer. >> > >> > Am I understanding it right. >> >> Like directories, we'd only need to set REDIRECT on rename. >> >> So when file has METACOPY, but not REDIRECT, we just fall through to >> next layer below one we are currently operating on. If we find >> METACOPY there, we just continue looking until we find a file >> containing the data. >> >> When we rename or hardlink a file with METACOPY, we add REDIRECT. >> >> If file has METACOPY and REDIRECT, we follow REDIRECT to find a file >> on the next level and keep iterating until we have the one with the >> data. >> >> ORIGIN would not be used in this case. We might be able to use ORIGIN >> for some kind of verification, like we do for directories. Amir has >> a better idea, I think. >> >> Another way to think about it is: METACOPY is the opposite of OPAQUE. >> For directories the default is "metacopy" and contents are merged. >> For files the default is "opaque" and content is not merged. METACOPY >> turns that around and enables "merging" of data from a lower layer. >> I could even imagine real merging of data, but it's unlikely to be >> worth the effort, clone is much better for that; METACOPY is just a >> very restricted (and so much simpler) way of merging data. > > Ok, thanks. I am beginning to understand it better now. > > First implementaion issue which comes to my mind is that stack[0] location > conflict. Right now this is taken up by dentry which was obtained by following > ORIGIN from upper and acts as copy up origin. > > May be I should continue to use ORIGIN for upper dentry and when stack[0] is > filled and if its metacopy, then continue to find data dentry using either > REDIRECT or using same name and store in stack[1]. > Question: don't you think it would be beneficial to get metacopy working and tested only from upper and without taking security considerations into the mix for first version? Do you know there is a real use case for middle layer metacopy and chaining and all that Jazz? When you first presented metacopy it sounded like you have a very solid use case (chown -R). Does your specific use case extend to middle layers? Is metacopy valueable enough without middle layers following? Heck, AFAIK, container runtime doesn't even know how to deal with redirect yet when committing an upper layer to an image. right? Just wondering... Amir. -- To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
