On May 25, 2021, at 4:31 PM, David Howells <dhowells@xxxxxxxxxx> wrote: > > Andreas Dilger <adilger@xxxxxxxxx> wrote: > >> As described elsewhere in the thread, allowing concurrent create and unlink >> in a directory (rename probably not needed) would be invaluable for scaling >> multi-threaded workloads. Neil Brown posted a prototype patch to add this >> to the VFS for NFS: > > Actually, one thing I'm looking at is using vfs_tmpfile() to create a new file > (or a replacement file when invalidation is required) and then using > vfs_link() to attach directory entries in the background (possibly using > vfs_link() with AT_LINK_REPLACE[1] instead of unlink+link). > > Any thoughts on how that might scale? vfs_tmpfile() doesn't appear to require > the directory inode lock. I presume the directory is required for security > purposes in addition to being a way to specify the target filesystem. I don't see how that would help much? Yes, the tmpfile allocation would be out-of-line vs. the directory lock, so this may reduce the lock hold time by some fraction, but this would still need to hold the directory lock when linking the tmpfile into the directory, in the same way that create and unlink are serialized against other threads working in the same dir. Having the directory locking scale with the size of the directory is what will get orders of magnitude speedups for large concurrent workloads. In ext4 this means write locking the directory leaf blocks independently, with read locks for the interior index blocks unless new leaf blocks are added (they are currently never removed). It's the same situation as back with the BKL locking the entire kernel, before we got fine-grained locking throughout the kernel. > > David > > [1] https://lore.kernel.org/linux-fsdevel/cover.1580251857.git.osandov@xxxxxx/ > Cheers, Andreas
Attachment:
signature.asc
Description: Message signed with OpenPGP
