On 07/11/2013 10:27 AM, Mihály Árva-Tóth wrote:
2013/7/11 Mark Nelson <mark.nelson@xxxxxxxxxxx <mailto:mark.nelson@xxxxxxxxxxx>> On 07/11/2013 10:04 AM, Mihály Árva-Tóth wrote: Hello, We are planning to use Intel 10 GE ethernet between nodes of OSDs. Host operation system will be Ubuntu 12.04 x86_64. Are there any recommendations available to tuning options (ex. sysctl and ceph)? Thank you, Mihaly Hi, Generally if performance and latency look good with something like iperf and a couple of parallel streams you should be able to get good performance with Ceph. You may find that using jumbo frames can help in some circumstances. In some cases we've seen that TCP autotuning can cause issues (primarily with reads!), but I think we've basically got that solved through a ceph tunable now. Hi Mark, Thank you. So are there no Ceph-related configration options which can I tuning for good performance on 10GE network? Where can I read more about TCP autotuning issues?
Nothing really comes to mind as far as Ceph goes. You may want to use a separate front and back network if you have the ports/switches available. Having said that, I've got a test setup where I used a bonded 10GbE interface, and with RADOS bench was able to achieve 2GB/s with no special Ceph network options beyond specifying that I wanted to use the 10GbE network. Of course you'll need the clients, concurrency, and backend disks to really get that.
The tcp autotuning issues were first discovered by Jim Schutt about a year ago and reported on ceph-devel:
http://www.spinics.net/lists/ceph-devel/msg05049.html And our workaround: http://tracker.ceph.com/issues/2100
Regards, Mihaly _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
