"J. Bruce Fields" <bfields@xxxxxxxxxxxx> wrote on 06/30/2008 09:05:41 PM:
> On Mon, Jun 30, 2008 at 11:26:54AM -0400, Jeff Layton wrote:
> > Recently I spent some time with others here at Red Hat looking
> > at problems with nfs server performance. One thing we found was that
> > there are some problems with multiple nfsd's. It seems like the I/O
> > scheduling or something is fooled by the fact that sequential write
> > calls are often handled by different nfsd's. This can negatively
> > impact performance (I don't think we've tracked this down completely
> > yet, however).
>
> Yes, we've been trying to see how close to full network speed we can get
> over a 10 gig network and have run into situations where increasing the
> number of threads (without changing anything else) seems to decrease
> performance of a simple sequential write.
>
> And the hypothesis that the problem was randomized IO scheduling was the
> first thing that came to mind. But I'm not sure what the easiest way
> would be to really prove that that was the problem.
>
> And then once we really are sure that's the problem, I'm not sure what
> to do about it. I suppose it may depend partly on exactly where the
> reordering is happening.
For 1 process, this theory seems to work:
1 testing process: /local: 39.11 MB/s
64 nfsd's: 29.63 MB/s
1 nfs'd: 38.99 MB/s
However for 6 processes reading 6 different files:
6 parallel testing processes: /local: 70 MB/s
1 nfs'd: 36 MB/s (49% drop)
2 nfs'd: 37.7 MB/s (46% drop)
4 nfs'd: 38.6 MB/s (44.9% drop)
4 nfsd's on different cpu's: 37.5 MB/s (46% drop)
32 nfs'd: 38.3 MB/s (45% drop)
64 nfs'd: 38.3 MB/s (45% drop)
Thanks,
- KK
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
