FYI, I found this in my notes. It gives a flavor of the
kind of lock contention we were discussing last week.
My client is a 12-core dual socket system. Networking
is 56Gbps IB, NFS/RDMA. I'm running some multi-threaded
iozone tests. Here I was mixing buffered and direct I/O
tests, so I can't say for certain if one is worse than
the other.
The output below is the first entry in /proc/lock_stat.
"sb->s_type->i_lock_key" looked like it was related to
the superblock, but looking at the actual code in the
functions named below, this is probably the inode lock.
The max lock holdtime is 82 milliseconds, if I read this
correctly. It's not clear how often that happens, but
that looks pathological.
lock_stat version 0.4
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
&sb->s_type->i_lock_key#2: 23185746 23275952 0.10 81771.03 27980172.31 1.20 81467297 238209945 0.09 81842.12 80513710.68 0.34
-------------------------
&sb->s_type->i_lock_key#2 1453378 [<ffffffffa06c6e49>] nfs_flush_incompatible+0x75/0x1ad [nfs]
&sb->s_type->i_lock_key#2 4777000 [<ffffffffa06c4ed6>] nfs_lock_and_join_requests+0x68/0x2a3 [nfs]
&sb->s_type->i_lock_key#2 157315 [<ffffffffa06c72f5>] nfs_updatepage+0x374/0x912 [nfs]
&sb->s_type->i_lock_key#2 2085447 [<ffffffffa06c74d7>] nfs_updatepage+0x556/0x912 [nfs]
-------------------------
&sb->s_type->i_lock_key#2 132 [<ffffffff8123cc2c>] writeback_sb_inodes+0xfb/0x50c
&sb->s_type->i_lock_key#2 7858928 [<ffffffffa06c4ed6>] nfs_lock_and_join_requests+0x68/0x2a3 [nfs]
&sb->s_type->i_lock_key#2 1225835 [<ffffffffa06c6e49>] nfs_flush_incompatible+0x75/0x1ad [nfs]
&sb->s_type->i_lock_key#2 111821 [<ffffffffa06c72f5>] nfs_updatepage+0x374/0x912 [nfs]
--
Chuck Lever
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
