2013/7/11 Mark Nelson <mark.nelson@xxxxxxxxxxx>
On 07/11/2013 10:27 AM, Mihály Árva-Tóth wrote:
2013/7/11 Mark Nelson <mark.nelson@xxxxxxxxxxx
<mailto:mark.nelson@inktank.com>>
On 07/11/2013 10:04 AM, Mihály Árva-Tóth wrote:
Hello,
We are planning to use Intel 10 GE ethernet between nodes of
OSDs. Host
operation system will be Ubuntu 12.04 x86_64. Are there any
recommendations available to tuning options (ex. sysctl and ceph)?
Thank you,
Mihaly
Hi,
Generally if performance and latency look good with something like
iperf and a couple of parallel streams you should be able to get
good performance with Ceph. You may find that using jumbo frames
can help in some circumstances. In some cases we've seen that TCP
autotuning can cause issues (primarily with reads!), but I think
we've basically got that solved through a ceph tunable now.
Hi Mark,
Thank you. So are there no Ceph-related configration options which can I
tuning for good performance on 10GE network? Where can I read more about
TCP autotuning issues?
Nothing really comes to mind as far as Ceph goes. You may want to use a separate front and back network if you have the ports/switches available. Having said that, I've got a test setup where I used a bonded 10GbE interface, and with RADOS bench was able to achieve 2GB/s with no special Ceph network options beyond specifying that I wanted to use the 10GbE network. Of course you'll need the clients, concurrency, and backend disks to really get that.
The tcp autotuning issues were first discovered by Jim Schutt about a year ago and reported on ceph-devel:
http://www.spinics.net/lists/ceph-devel/msg05049.html
And our workaround:
http://tracker.ceph.com/issues/2100
Hi Mark,
Thank you, great news.
Regards,
Mihaly
Thank you, great news.
Regards,
Mihaly
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
