---- 在 星期四, 2020-10-15 14:11:04 Amir Goldstein <amir73il@xxxxxxxxx> 撰写 ---- > On Thu, Oct 15, 2020 at 6:03 AM Chengguang Xu <cgxu519@xxxxxxxxxxxx> wrote: > > > > ---- 在 星期四, 2020-10-15 00:15:38 Jan Kara <jack@xxxxxxx> 撰写 ---- > > > On Sat 10-10-20 22:23:51, Chengguang Xu wrote: > > > > Currently there is no notification api for kernel about modification > > > > of vfs inode, in some use cases like overlayfs, this kind of notification > > > > will be very helpful to implement containerized syncfs functionality. > > > > As the first attempt, we introduce marking inode dirty notification so that > > > > overlay's inode could mark itself dirty as well and then only sync dirty > > > > overlay inode while syncfs. > > > > > > > > Signed-off-by: Chengguang Xu <cgxu519@xxxxxxxxxxxx> > > > > > > So I like how the patch set is elegant however growing struct inode for > > > everybody by struct blocking_notifier_head (which is rwsem + pointer) is > > > rather harsh just for this overlayfs functionality... Ideally this should > > > induce no overhead on struct inode if the filesystem is not covered by > > > overlayfs. So I'd rather place some external structure into the superblock > > > that would get allocated on the first use that would track dirty notifications > > > for each inode. I would not generalize the code for more possible > > > notifications at this point. > > > > > > Also now that I'm thinking about it can there be multiple overlayfs inodes > > > for one upper inode? If not, then the mechanism of dirtiness propagation > > > > One upper inode only belongs to one overlayfs inode. However, in practice > > one upper fs may contains hundreds or even thousands of overlayfs instances. > > > > > could be much simpler - it seems we could be able to just lookup > > > corresponding overlayfs inode based on upper inode and then mark it dirty > > > (but this would need to be verified by people more familiar with > > > overlayfs). So all we'd need to know for this is the superblock of the > > > overlayfs that's covering given upper filesystem... > > > > > > > So the difficulty is how we get overlayfs inode efficiently from upper inode, > > it seems if we don't have additional info of upper inode to indicate which > > overlayfs it belongs to, then the lookup of corresponding overlayfs inode > > will be quite expensive and probably impact write performance. > > > > Is that possible we extend inotify infrastructure slightly to notify both > > user space and kernel component? > > > > > > When I first saw your suggestion, that is what I was thinking. > Add event fsnotify_dirty_inode(), since the pub-sub infrastructure > in struct inode already exists. > > But I have to admit this approach seems like a massive overkill to > what you need. > > What you implemented, tracks upper inodes that could have > been dirtied under overlayfs, but what you really want is to > track is upper inodes that were dirtied *by* overlayfs. > > And for that purpose, as Miklos said several times, it would be best > pursue the overlayfs aops approach, even though it may be much > harder.. > IIUC, that solution was raised to solve mmap rw-ro issue. That maybe is an ultimate goal and I'm wondering whether we must implement that if we have easier approach to solve mmap and syncfs issues. > Your earlier patches maintained a list of overlayfs to be synced inodes. > Remind me what was wrong with that approach? > I think the main concerns are the complexity and the timing of releasing ovl_sync_entry struct. > Perhaps you can combine that with the shadow overlay sbi approach. > Instead of dirtying overlay inode when underlying is dirtied, you can > "pre-dirty" overlayfs inode in higher level file ops and add them to the > "maybe-dirty" list (e.g. after write). > Main problem is we can't be notified by set_page_dirty from writable mmap. Meanwhile, if we dirty overlay inode then writeback will pick up dirty overlay inode and clear it after syncing, then overlay inode could be release at any time, so in the end, maybe overlay inode is released but upper inode is still dirty and there is no any pointer to find upper dirty inode out. > ovl_sync_fs() can iterate the maybe-dirty list and re-dirty overlay inodes > if the underlying inode is still dirty on the (!wait) pass. > > As for memory mapped inodes via overlayfs (which can be dirtied without > notifying overlayfs) I am not sure that is a big problem in practice. > Yes, it's key problem here. > When an inode is writably mapped via ovarlayfs, you can flag the > overlay inode with "maybe-writably-mapped" and then remove > it from the maybe dirty list when the underlying inode is not dirty > AND its i_writecount is 0 (checked on write_inode() and release()). > > Actually, there is no reason to treat writably mapped inodes and > other dirty inodes differently - insert to suspect list on open for > write, remove from suspect list on last release() or write_inode() > when inode is no longer dirty and writable. > > Did I miss anything? > If we dirty overlay inode that means we have launched writeback mechanism, so in this case, re-dirty overlay inode in time becomes important. Thanks, Chengguang
