I've personally saturated 1Gbps links on multiple radosgw nodes on a large cluster, if I remember correctly, Yehuda has tested it up into the 7Gbps range with 10Gbps gear. Could you describe your clusters hardware and connectivity?
--
Kyle
On Mon, Oct 14, 2013 at 3:34 AM, Chu Duc Minh <chu.ducminh@xxxxxxxxx> wrote:
Hi sorry, i missed this mail.No, CPU stay the same & very low (< 10%)
> During writes, does the CPU usage on your RadosGW node go way up?When upload small files(300KB/file) over RadosGW:- using 1 process: upload bandwidth ~ 3MB/s- using 100 processes: upload bandwidth ~ 15MB/s
When upload big files(3GB/file) over RadosGW:- using 1 process: upload bandwidth ~ 70MB/s(Therefore i don't upload big files using multi-processes any more :D)
Maybe, RadosGW have a problem when write many smail files. Or it's a problem of CEPH when simultaneously write many smail files into a bucket, that already have millions files?On Wed, Sep 25, 2013 at 7:24 PM, Mark Nelson <mark.nelson@xxxxxxxxxxx> wrote:
On 09/25/2013 02:49 AM, Chu Duc Minh wrote:
I have a CEPH cluster with 9 nodes (6 data nodes & 3 mon/mds nodes)_*Problem*_: i can pass limit 50MB/s when put file over Rados-GW,
And i setup 4 separate nodes to test performance of Rados-GW:
- 2 node run Rados-GW
- 2 node run multi-process put file to [multi] Rados-GW
Result:
a) When i use 1 RadosGW node & 1 upload-node, speed upload = 50MB/s
/upload-node, Rados-GW input/output speed = 50MB/s
b) When i use 2 RadosGW node & 1 upload-node, speed upload = 50MB/s
/upload-node; each RadosGW have input/output = 25MB/s ==> sum
input/ouput of 2 Rados-GW = 50MB/s
c) When i use 1 RadosGW node & 2 upload-node, speed upload = 25MB/s
/upload-node ==> sum output of 2 upload-node = 50MB/s, RadosGW have
input/output = 50MB/s
d) When i use 2 RadosGW node & 2 upload-node, speed upload = 25MB/s
/upload-node ==> sum output of 2 upload-node = 50MB/s; each RadosGW have
input/output = 25MB/s ==> sum input/ouput of 2 Rados-GW = 50MB/s
regardless of the number Rados-GW nodes and upload-nodes.
When i use this CEPH cluster over librados (openstack/kvm), i can easily
achieve > 300MB/s
I don't know why performance of RadosGW is so low. What's bottleneck?
During writes, does the CPU usage on your RadosGW node go way up?
If this is a test cluster, you might want to try the wip-6286 build from our gitbuilder site. There is a fix that depending on the size of your objects, could have a big impact on performance. We're currently investigating some other radosgw performance issues as well, so stay tuned. :)
Mark
Thank you very much!
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Kyle
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
