On Fri, 2 Jul 2010 10:12:47 -0600 Andreas Dilger <andreas.dilger@xxxxxxxxxx> wrote: > On 2010-07-02, at 01:05, hch@xxxxxxxxxxxxx wrote: > > On Thu, Jul 01, 2010 at 10:02:29PM -0600, Andreas Dilger wrote: > >> I'd like to be able to use this interface to implement the distributed open call proposed by the POSIX HECWG. This allows one client to do the path traversal, broadcast the file handle to the (maybe) 1M processes in the job via MPI, and then the other clients can open the file by handle without doing 1M times the full path traversal (which might be 10's of RPCs per process). > > > > The proposal is doomed anyway. If we allow any sort of open by handle > > system call for unprivilegued users we need to do reconnect the dentry > > to the dcache path anyway (reconnect_path), which is more expensive than > > a normal path lookup. > > I haven't looked at this part of the VFS in a while, but it looks like an implementation issue specific to knfsd, and shouldn't be needed for regular files. i.e. if exportfs_encode_fh() is never used on a disconnected file, then this overhead is not incurred. > > The above use of open_by_handle() is not for userspace NFS/Samba re-export, but to allow applications to open regular files for IO. > From my recollection of implementing dentry reconnection there are two needs for it. Firstly it is needed for directories so that the VFS can effectively lock against directory rename races which could otherwise create disconnected subtrees (where the first parent is a member only of one of its descendants). So if you get a filehandle for a directory it *must* be properly connected to the root for rename to be safe. This operation is faster than a full path lookup if the dentry is already is cache, and slower if it and any of the path is not in cache. You could possibly delay the full-connection of the dentry until the first attempt to rename beneath it. I'm not sure how much VFS surgery that would require. Secondly it is needed if you want to enforce the rule that the contents of a directory are only accessible if the 'x' bit on the directory is set. kNFSd does not enforce this (unless subtree_check is specified), partly because it is hard to do correctly and partly because we have to trust the client any, so trusting it to check the 'x' bit is very little extra trust. Note that it is not possible to reliably perform filehandle lookup for non-directories if you need a fully reconnected dentry, as cross-directory-renames can confuse the situation beyond recovery. Maybe open-by-handle should require DAC_OVERRIDE, or maybe a new DAC_X_OVERRIDE. And if those aren't provided it only works for directories. ??? NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
