Hi Zheng,Interesting analysis.
* On Sun, Jan 15, 2012 at 03:17:12PM -0500, Zheng Da <zhengda1936@xxxxxxxxx> wrote:
From what I have heard it has supported it from sometime back. I think you may need to ask in xfs general ML about this.Thanks. I was reading the code of kernel 3.0. XFS starts to support concurrent direct IO since kernel 3.1.5.But concurrent direct IO write still doesn't work well in kernel 3.2.
I wrote a test program that accesses a 4G file randomly (read and write), and
I ran it with 8 threads and the machine has 8 cores. It turns out that only
1 core is running. I'm pretty sure xfs_rw_ilock is locked
with XFS_IOLOCK_SHARED in xfs_file_dio_aio_write.
lockstat shows me that there is a lot of wait time in ip->i_lock. It seems
the lock is locked exclusively.
&(&ip->i_lock)->mr_lock-W: 31568 36170
0.24 20048.25 7589157.99 130154 3146848
0.00 217.70 1238310.72
&(&ip->i_lock)->mr_lock-R: 11251 11886
0.24 20043.01 2895595.18 46671 526309
0.00 63.80 264097.96
-------------------------
&(&ip->i_lock)->mr_lock 36170
[<ffffffffa03be122>] xfs_ilock+0xb2/0x110 [xfs]
&(&ip->i_lock)->mr_lock 11886
[<ffffffffa03be15a>] xfs_ilock+0xea/0x110 [xfs]
-------------------------
&(&ip->i_lock)->mr_lock 38555
[<ffffffffa03be122>] xfs_ilock+0xb2/0x110 [xfs]
&(&ip->i_lock)->mr_lock 9501
[<ffffffffa03be15a>] xfs_ilock+0xea/0x110 [xfs]
Then I used systemtap to instrument xfs_ilock and find there are at least 3
functions that lock ip->i_lock exclusively during write.
From what I saw in xfs_file_dio_aio_write code, it uses EXCL only if there is unaligned IO or there are cached pages to be invalidated after shared lock is obtained *but* it demotes that lock to SHARED just before generic_file_direct_write.
Is there any popular FS that supports concurrent direct IO well? Thanks, Da On Sat, Jan 14, 2012 at 6:45 AM, Raghavendra D Prabhu < raghu.prabhu13@xxxxxxxxx> wrote:Hi Zheng,
* On Fri, Jan 13, 2012 at 04:41:16PM -0500, Zheng Da < zhengda1936@xxxxxxxxx> wrote:
Hello,
I'm looking for a FS in Linux that supports concurrent direct IO write. ext4 supports concurrent direct IO read if we mount it with dioread_nolock, but doesn't support concurrent writes. XFS doesn't support concurrent direct IO at all. It locks the inode exclusive if it's direct IO. I tried btrfs, and it seems it doesn't support concurrent direct IO either though I haven't looked into its code. Is there a local FS that support concurrent direct IO write? It seems NFS supports it ( http://kevinclosson.wordpress.**com/2011/08/12/file-systems-** for-a-database-choose-one-**that-couples-direct-io-and-** concurrent-io-whats-this-have-**to-do-with-nfs-harken-back-5-** 2-years-to-find-out/<http://kevinclosson.wordpress.com/2011/08/12/file-systems-for-a-database-choose-one-that-couples-direct-io-and-concurrent-io-whats-this-have-to-do-with-nfs-harken-back-5-2-years-to-find-out/> ), but I'm looking for local FS.
Thanks, Da
______________________________**_________________Kernelnewbies mailing list Kernelnewbies@kernelnewbies.**org <Kernelnewbies@xxxxxxxxxxxxxxxxx> http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
XFS locks inode exclusive only if it is an unaligned Direct IO, which is apparently done to prevent race conditions -- refer to this http://oss.sgi.com/archives/**xfs/2011-01/msg00157.html<http://oss.sgi.com/archives/xfs/2011-01/msg00157.html>Also the behavior of Ext4 under dioread_nolock is supported by XFS by default and in a much better way. Also Ext4 is the only one which uses DIO_LOCKING while doing direct io.
Regards, -- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net
Regards, -- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net
Attachment:
pgp7FK_Jma6mM.pgp
Description: PGP signature
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@xxxxxxxxxxxxxxxxx http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
