Olga Kornievskaia wrote:
Trond Myklebust wrote:
On Wed, 2008-06-04 at 12:40 -0400, Olga Kornievskaia wrote:
While testing NFSv4 performance over the 10GE network, we are seeing
the following behavior and would like to know if it is normal or a
bug in the client code.
The server offers the max_write of 1M. The client mounts the server
with the "wsize" option of 1M. Yet during the write we are seeing
that the write size is at most 49K. Why does client never come close
to 1M limit?
Does /proc/mounts indicate 1M? As a total guess, could there be
something going on in nfs_can_coalesce_requests
</lxr-pnfs/ident?i=nfs_can_coalesce_requests>? (I can't imagine why,
but we had a pnfs problem where our additions to
nfs_can_coalesce_requests were causing similar behavior)
I have a feeling that is due to some crap in the VM. I'm currently
investigating a situation where it appears we're sending 1 COMMIT for
every 1-5 32k WRITEs. This is not a policy that stems from the NFS
client, so it would appear that the VM is being silly about things.
I'm specially suspicious of the code in get_dirty_limits() that is
setting a limit to the number of dirty pages based on the number of
pages a given BDI has written out in the recent past. As far as I can
see, the intention is to penalise devices that are slow writers, but in
practice it doesn't do that: it penalises the devices that have the
least activity.
I think we are seeing larger than usual number of COMMIT messages.
With older kernels, no matter how I configured linux to flush dirty
pages, it would always flush too often and hence send way too many
commit messages. But more like every couple hundred megabytes.....
If you don't need the page cache, I have found that O_DIRECT can
increase performance by giving predefined points at which the client
will commit.
Dean
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
