Dean wrote: On 4/24/14, 10:22 AM, Cedric Blancher wrote: >On 24 April 2014 05:12, Jim Rees <rees@xxxxxxxxx> wrote: >>Cedric Blancher wrote: >> >> Are there any options to improve the Linux NFSv4 performance over a >> high latency connection? >> >>We did some work along these lines at CITI years ago. As I remember, the >>main thing was to increase net.ipv4.tcp_[rw]mem on the server side, because >>tcp auto-tuning was being defeated. This may be less of an issue with your >>work load, which sounds like many small files rather than one big one. In >>theory, NFSv4 delegations should help, but I don't know how well that works. Along with Jim's work, we followed up with a fair bit, but in general we found that nfs clients just can't do well over large rtt due to the slow window ramp up time and adverse reaction to packet loss. Unfortunately the only way to overcome these issues (other than using a custom udp protocol which isn't supported) is to use multiple TCP connections, which is what we do by using multiple nodes.... Yeah, at the time I think reno was the default congestion, and you need something with a faster rampup. I believe cubic is default now and it's pretty good but still not good enough. Andy Adamson did some work too, making the number of rpc slots dynamic, and I think that's in the kernel now. If you've got a very high speed network, like say 10Gb with >100 msec, you may need to do some tuning in the ethernet driver, increasing ring buffer sizes and so on. Your congestion window can grow to hundreds of MB in this case. And there's no getting around that nfs is fairly chatty. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
