Steven Whitehouse wrote: > Hi, > > On Thu, 2013-08-22 at 16:05 +0300, Kirill A. Shutemov wrote: > > Steven Whitehouse wrote: > > > Hi, > > > > > > On Wed, 2013-08-21 at 13:58 -0700, Andrew Morton wrote: > > > > On Wed, 21 Aug 2013 17:42:12 +0100 Steven Whitehouse <swhiteho@xxxxxxxxxx> wrote: > > > > > > > > > > I don't think the change is harmful. The worst case scenario is race with > > > > > > write or truncate, but it's valid to return EOF in this case. > > > > > > > > > > > > What scenario do you have in mind? > > > > > > > > > > > > > > > > 1. File open on node A > > > > > 2. Someone updates it on node B by extending the file > > > > > 3. Someone reads the file on node A beyond end of original file size, > > > > > but within end of new file size as updated by node B. Without the patch > > > > > this works, with it, it will fail. The reason being the i_size would not > > > > > be up to date until after readpage(s) has been called. > > > > CC: +linux-fsdevel@ > > > > So in this case node A will see the file like it was never touched by > > node B. It's okay, if new i_size will eventually reach node A. > > > > Is ->readpage() the only way to get i_size updated on node A or it will be > > eventually updated without it? > > > It will be updated by anything which takes a glock in the gfs2 case. > Note that ->readpage() is not a very frequently used aop. For any > filesystem with ->readpages() this should cover almost all calls to the > fs for reading, with ->readpage() only used for some corner cases. So > from a performance point of view, it is ->readpages() which matters > most. > > There is no time based updating of the inode information - it relies > entirely upon the locking/cache control provided by the glock layer. > > > If it's the only way, we need add a explicit way to initiate i_size sync > > between nodes on read. Probably, distributed filesystems should provide own > > ->aio_read() which deal i_size as the filesystem need. > > > I'd rather not do that, if we can avoid it. The current system has been > carefully designed so that all the cluster fs knowledge can be hidden in > the layer below the page cache. That means for a read which can be > satisfied from just the page cache, cluster filesystems are the same > speed as local file systems, since it is the same code path. Only when a > page doesn't exist in the cache do we need to take cluster locks in > order to check the file size, etc. > > Taking cluster locks can be expensive, since in the worst case it can > involve both the local glock and dlm state machines, and remote dlm and > glock state machines, network communication, and waiting for disk i/o > and log flushes on (a) remote node(s). > > Andrew's proposed solution makes sense to me, and is probably the > easiest way to solve this. Move check to no_cached_page? I don't see how it makes any difference for page cache miss case: we anyway exclude ->readpage() if it's beyond local i_size. And for cache hit local i_size will be most likely cover locally cached pages. Should we introduce an aop which can be called before i_size check in no_cached_page path to refresh local i_size? > > > > > I think this is likely to be an issue for any distributed fs using > > > > > do_generic_file_read(), although it would certainly affect GFS2, since > > > > > the locking is done at page cache level, > > > > > > > > Boy, that's rather subtle. I'm surprised that the generic filemap.c > > > > stuff works at all in that sort of scenario. > > > > > > > > Can we put the i_size check down in the no_cached_page block? afaict > > > > that will solve the problem without breaking GFS2 and is more > > > > efficient? > > > > > > > > > > Well I think is even more subtle, since it relies on ->readpages > > > updating the file size, even if it has failed to actually read the > > > required pages :-) Having said that, we do rely on ->readpages updating > > > the inode size elsewhere in this function, as per the block comment > > > immediately following the page_ok label. > > > > That i_size recheck was invented to cover different use case: read vs. > > truncate race. Userspace should not see truncate-caused zeros in buffer. > > It's not to prevent file extending vs. read() race. This usually harmless: > > data is consistent. > > > > Yes, it is a different use case, but the same issue applies that without > a prior call to ->readpage() or ->readpages() then i_size may not be > correct. I believe it's correct, but stale. I think it makes difference in the context. Use stale value for read vs. write race is okay, but not for read vs. truncate. Will i_size be set to correct value on the first file open (struct inode creation)? > The comment mentions that there needs to be an uptodate page in > existence in order for i_size to be valid, and that is, at least in the > GFS2 case, an equivalent condition since it implies that ->readpage() or > ->readpages() must have been called since the address space was last > invalidated (or alternatively that the page was populated from a local > write, but the effect is the same) > > This is only true though, since GFS2 does its locking/caching on a > per-inode basis - it someone wanted to implement a filesystem which > worked on a per-page (for example) basis then the uptodate page criteria > would not necessarily mean that the i_size was also uptodate. > > I hope that helps clarify things a bit, > > Steve. -- Kirill A. Shutemov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>