---- 在 星期日, 2020-03-29 23:06:42 Amir Goldstein <amir73il@xxxxxxxxx> 撰写 ---- > On Sun, Mar 29, 2020 at 5:19 PM Chengguang Xu <cgxu519@xxxxxxxxxxxx> wrote: > > > > ---- 在 星期五, 2020-03-27 17:45:37 Amir Goldstein <amir73il@xxxxxxxxx> 撰写 ---- > > > On Fri, Mar 27, 2020 at 8:18 AM Chengguang Xu <cgxu519@xxxxxxxxxxxx> wrote: > > > > > > > > ---- 在 星期四, 2020-03-26 15:34:13 Amir Goldstein <amir73il@xxxxxxxxx> 撰写 ---- > > > > > On Thu, Mar 26, 2020 at 7:45 AM Chengguang Xu <cgxu519@xxxxxxxxxxxx> wrote: > > > > > > > > > > > > Hello, > > > > > > > > > > > > On container use case, in order to prevent inode exhaustion on host file system by particular containers, we would like to add inode limitation for containers. > > > > > > However, current solution for inode limitation is based on project quota in specific underlying filesystem so it will also count deleted files(char type files) in overlay's upper layer. > > > > > > Even worse, users may delete some lower layer files for getting more usable free inodes but the result will be opposite (consuming more inodes). > > > > > > > > > > > > It is somewhat different compare to disk size limitation for overlayfs, so I think maybe we can add a limit option just for new files in overlayfs. What do you think? > > > > > > You are saying above that the goal is to prevent inode exhaustion on > > > host file system, > > > but you want to allow containers to modify and delete unlimited number > > > of lower files > > > thus allowing inode exhaustion. I don't see the logic is that. > > > > > > > End users do not understand kernel tech very well, so we just want to mitigate > > container's different user experience as much as possible. In our point of view, > > consuming more inode by deleting lower file is the feature of overlayfs, it's not > > caused by user's abnormal using. However, we have to limit malicious user > > program which is endlessly creating new files until host inode exhausting. > > > > > > > Even if we only count new files and present this information on df -i > > > how would users be able to free up inodes when they hit the limit? > > > How would they know which inodes to delete? > > > > > > > > > > > > > The questions are where do we store the accounting and how do we maintain them. > > > > > An answer to those questions could be - in the inode index: > > > > > > > > > > Currently, with nfs_export=on, there is already an index dir containing: > > > > > - 1 hardlink per copied up non-dir inode > > > > > - 1 directory per copied-up directory > > > > > - 1 whiteout per whiteout in upperdir (not an hardlink) > > > > > > > > > > > > > Hi Amir, > > > > > > > > Thanks for quick response and detail information. > > > > > > > > I think the simplest way is just store accounting info in memory(maybe in s_fs_info). > > > > At very first, I just thought doing it for container use case, for container, it will be > > > > enough because the upper layer is always empty at starting time and will be destroyed > > > > at ending time. > > > > > > That is not a concept that overlayfs is currently aware of. > > > *If* the concept is acceptable and you do implement a feature intended for this > > > special use case, you should verify on mount time that upperdir is empty. > > > > > > > > > > > Adding a meta info to index dir is a better solution for general use case but it seems > > > > more complicated and I'm not sure if there are other use cases concern with this problem. > > > > Suggestion? > > > > > > docker already supports container storage quota using project quotas > > > on upperdir (I implemented it). > > > Seems like a very natural extension to also limit no. of inodes. > > > The problem, as you wrote it above is that project quotas > > > "will also count deleted files(char type files) in overlay's upper layer." > > > My suggestion to you was a way to account for the whiteouts separately, > > > so you may deduct them from total inode count. > > > If you are saying my suggestion is complicated, perhaps you did not > > > understand it. > > > > > > > I think the key point here is the count of whiteout inode. I would like to > > propose share same inode with different whiteout files so that we can save > > inode significantly for whiteout files. After this, I think we can just implement > > normal inode limit for container just like block limit. > > > > Very good idea. See: > https://lore.kernel.org/linux-unionfs/20180301064526.17216-1-houtao1@xxxxxxxxxx/ > > I don't think Tao ever followed up with v3 patch. > Thanks Amir, the feature looks exactly what we need. Hi Tao, Would you have plan to update the patch "overlay: hardlink all whiteout files into a single one" in the recent future? I'm trying to fix a issue regarding to inode limitation in container and inode sharing with whiteout files is the key point to support the solution. Thanks, cgxu
