On 06/11/2013 08:27 AM, Jäger, Philipp wrote:
Hello, we have a performance problem with radosrgw. Only 8mb/s-9 per upload, also tested with s3cmd on the rgw itself. (2 uploads at the same time: combined 15mb/s, 3 uploads at the same time: comb. 21mb/s) But when putting a file via rados rbd , we get 40mb/s upload, so no network or other problem in general.
One thing to check is to make sure that the rgw pool you are writing to has enough placement groups for your cluster. The default may be extremely low.
Same speed with the inktank apache/fastcgi and the original one. Hardware also fast enough. We use Ubuntu 12.04 lts, ceph 0.61.2 So have you any idea why the rgw is so slow? How can we identify where the problem is?
RBD is pretty streamlined so you can get good performance with it. On my test setup I'm seeing 80-90% of the performance of raw rados object writes/reads (and in some cases much faster with RBD cache enabled!). RGW, Apache, fastcgi, and simply the requirements of supporting the S3 protocol itself add a lot of overhead. MD5 calculations by themselves start chewing up a ton of CPU once you try to support high throughput scenarios and there is a non-trivial amount of extra latency added as well. You may be able to improve things with some tweaks, but I wouldn't be surprised if RBD is always going to be faster to an extent.
For folks who want really fast object storage I think directly utilizing rados is probably the way to go, but that requires modifying the app and it's not for everyone.
(I've heard something about the rgw admin socket to check perfcounters, but it seems that this is deprecated? Because when i type ceph --admin-daemon ... it says unknown command and I cannot find it in the ceph docu. Then i wanted to bench via rest-bench, but it says "ERROR: failed to create bucket: XmlParseFailure -failed initializing benchmark", so I could not bench the speed.)
connecting with the admin daemon should still be supported. Documentation is here:
http://ceph.com/docs/next/radosgw/troubleshooting/ If this doesn't work please let me know! Also, I've created a bug for the rest-bench issue: http://tracker.ceph.com/issues/5302 Personally I've been using swift-bench for most of my recent rgw testing. Mark
Ceph.conf- rgw part: [client.radosgw.connect2] host = hcrgwko2 rgw socket path = /tmp/connect2.sock log file = /var/log/ceph/connect2.log rgw dns name = FQDN Thank you very much. Regards Philipp -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
-- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
