-Sigh- Yehuda, thanks for the hint. Didn't even have to debug the OSDs, I found out which OSD the initial request went to and looked at the logs and saw this: _load_class could not open class /usr/lib64/rados-classes/libcls_rgw.so (dlopen failed): /usr/lib64/rados-classes/libcls_rgw.so: cannot open shared object file: No such file or directory That directory has libcls_rgw.so.1.0.0, but apparently the OSD is just looking for the .so file. So, I created the necessary symlink on all of my OSDs. Time to fix my RPM SPEC file, now. Thanks! -Nick >>> On 2012/09/03 at 04:44, Yehuda Sadeh <yehuda@xxxxxxxxxxx> wrote: > Error 5 means EIO. Can you set 'debug ms = 1' on the radosgw (and > probably also on the osd to correlate)? > > Thanks, > Yehuda > > On Mon, Sep 3, 2012 at 12:56 AM, Nick Couchman <Nick.Couchman@xxxxxxxxx> > wrote: >> So, after much trial and error I finally got ceph up and running and radosgw > up and running. However, I'm not running into a situation where, when I try > to create a new bucket under my test account, I'm getting an HTTP status 500 > on the client. I have radosgw running in debug mode, and I get the below > response. I'm not exactly sure how to interpret that - it looks to me like > ceph gets the response, verifies credentials and access, verifies that a pool > is available (.rgw.buckets), but then, for some reason runs into a return > value of -5 on the "rgw_create_bucket" function. I don't know what that value > represents, but the next line has err_no=5, then it says that it's returning > http status 500. Any hints as to what return value of -5 means, error code 5, > or why I'm getting an internal error trying to create a bucket? >> >> Thanks - Nick >> >> 2012-09-03 01:45:51.757251 7f8aa97f2700 1 ====== starting new request > req=0xf62550 ===== >> 2012-09-03 01:45:51.757299 7f8aa97f2700 2 req 20:0.000048::::initializing >> 2012-09-03 01:45:51.757320 7f8aa97f2700 10 s->object= s->bucket=test >> 2012-09-03 01:45:51.757329 7f8aa97f2700 20 FCGI_ROLE=RESPONDER >> 2012-09-03 01:45:51.757331 7f8aa97f2700 20 SCRIPT_URL=/test >> 2012-09-03 01:45:51.757334 7f8aa97f2700 20 > SCRIPT_URI=http://cloud.seakr.com/test >> 2012-09-03 01:45:51.757336 7f8aa97f2700 20 HTTP_AUTHORIZATION=AWS > BEH8OJ2EM7Z2WDAFFRPI:4qdgNWUJ4+NBK+tMoK2FheFKuUg= >> 2012-09-03 01:45:51.757338 7f8aa97f2700 20 RGW_DNS_NAME=cloud. >> 2012-09-03 01:45:51.757340 7f8aa97f2700 20 > HTTP_USER_AGENT=CloudBerryLab.Base.HttpUtil.Client 3.5.1 > (http://www.cloudberrylab.com/) >> 2012-09-03 01:45:51.757344 7f8aa97f2700 20 HTTP_DATE=Mon, 03 Sep 2012 07:45:52 > GMT >> 2012-09-03 01:45:51.757346 7f8aa97f2700 20 HTTP_HOST=cloud.seakr.com >> 2012-09-03 01:45:51.757348 7f8aa97f2700 20 CONTENT_LENGTH=0 >> 2012-09-03 01:45:51.757350 7f8aa97f2700 20 HTTP_CONNECTION=Close >> 2012-09-03 01:45:51.757352 7f8aa97f2700 20 > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin >> 2012-09-03 01:45:51.757355 7f8aa97f2700 20 SERVER_SIGNATURE= >> 2012-09-03 01:45:51.757357 7f8aa97f2700 20 SERVER_SOFTWARE=Apache/2.2.21 > (Linux/SUSE) >> 2012-09-03 01:45:51.757359 7f8aa97f2700 20 SERVER_NAME=cloud.seakr.com >> 2012-09-03 01:45:51.757361 7f8aa97f2700 20 SERVER_ADDR=192.168.30.102 >> 2012-09-03 01:45:51.757363 7f8aa97f2700 20 SERVER_PORT=80 >> 2012-09-03 01:45:51.757365 7f8aa97f2700 20 REMOTE_ADDR=192.168.10.24 >> 2012-09-03 01:45:51.757368 7f8aa97f2700 20 DOCUMENT_ROOT=/srv/www/htdocs >> 2012-09-03 01:45:51.757370 7f8aa97f2700 20 SERVER_ADMIN=[no address given] >> 2012-09-03 01:45:51.757372 7f8aa97f2700 20 > SCRIPT_FILENAME=/srv/www/htdocs/s3gw.fcgi >> 2012-09-03 01:45:51.757374 7f8aa97f2700 20 REMOTE_PORT=51971 >> 2012-09-03 01:45:51.757376 7f8aa97f2700 20 GATEWAY_INTERFACE=CGI/1.1 >> 2012-09-03 01:45:51.757378 7f8aa97f2700 20 SERVER_PROTOCOL=HTTP/1.1 >> 2012-09-03 01:45:51.757380 7f8aa97f2700 20 REQUEST_METHOD=PUT >> 2012-09-03 01:45:51.757383 7f8aa97f2700 20 QUERY_STRING=page=test*ms= >> 2012-09-03 01:45:51.757385 7f8aa97f2700 20 REQUEST_URI=/test >> 2012-09-03 01:45:51.757387 7f8aa97f2700 20 SCRIPT_NAME=/test >> 2012-09-03 01:45:51.757392 7f8aa97f2700 2 req 20:0.000140:s3:PUT > /test::getting op >> 2012-09-03 01:45:51.757398 7f8aa97f2700 2 req 20:0.000147:s3:PUT > /test:create_bucket:authorizing >> 2012-09-03 01:45:51.757417 7f8aa97f2700 20 get_obj_state: rctx=0x7f8ac0022540 > obj=.users:BEH8OJ2EM7Z2WDAFFRPI state=0x7f8ac00225f8 s->prefetch_data=0 >> 2012-09-03 01:45:51.757429 7f8aa97f2700 10 moving .users+BEH8OJ2EM7Z2WDAFFRPI > to cache LRU end >> 2012-09-03 01:45:51.757433 7f8aa97f2700 10 cache get: > name=.users+BEH8OJ2EM7Z2WDAFFRPI : hit >> 2012-09-03 01:45:51.757440 7f8aa97f2700 20 get_obj_state: s->obj_tag was set > empty >> 2012-09-03 01:45:51.757447 7f8aa97f2700 10 moving .users+BEH8OJ2EM7Z2WDAFFRPI > to cache LRU end >> 2012-09-03 01:45:51.757450 7f8aa97f2700 10 cache get: > name=.users+BEH8OJ2EM7Z2WDAFFRPI : hit >> 2012-09-03 01:45:51.757484 7f8aa97f2700 10 get_canon_resource(): dest=/test >> 2012-09-03 01:45:51.757488 7f8aa97f2700 10 auth_hdr: >> PUT >> >> >> Mon, 03 Sep 2012 07:45:52 GMT >> /test >> 2012-09-03 01:45:51.757552 7f8aa97f2700 15 b64=4qdgNWUJ4+NBK+tMoK2FheFKuUg= >> 2012-09-03 01:45:51.757556 7f8aa97f2700 15 > auth_sign=4qdgNWUJ4+NBK+tMoK2FheFKuUg= >> 2012-09-03 01:45:51.757558 7f8aa97f2700 15 compare=0 >> 2012-09-03 01:45:51.757563 7f8aa97f2700 2 req 20:0.000311:s3:PUT > /test:create_bucket:reading permissions >> 2012-09-03 01:45:51.757569 7f8aa97f2700 2 req 20:0.000317:s3:PUT > /test:create_bucket:verifying op permissions >> 2012-09-03 01:45:51.759125 7f8aa97f2700 2 req 20:0.001873:s3:PUT > /test:create_bucket:verifying op params >> 2012-09-03 01:45:51.759131 7f8aa97f2700 2 req 20:0.001880:s3:PUT > /test:create_bucket:executing >> 2012-09-03 01:45:51.759147 7f8aa97f2700 20 get_obj_state: rctx=0x7f8ac0013060 > obj=.rgw:test state=0x7f8ac00237c8 s->prefetch_data=0 >> 2012-09-03 01:45:51.759155 7f8aa97f2700 10 moving .rgw+test to cache LRU end >> 2012-09-03 01:45:51.759158 7f8aa97f2700 10 cache get: name=.rgw+test : type > miss (requested=6, cached=0) >> 2012-09-03 01:45:51.760342 7f8aa97f2700 10 cache put: name=.rgw+test >> 2012-09-03 01:45:51.760346 7f8aa97f2700 10 moving .rgw+test to cache LRU end >> 2012-09-03 01:45:51.760365 7f8aa97f2700 20 get_obj_state: rctx=0x7f8ac0018a00 > obj=.rgw:.pools.avail state=0x7f8ac0028068 s->prefetch_data=0 >> 2012-09-03 01:45:51.760372 7f8aa97f2700 10 moving .rgw+.pools.avail to cache > LRU end >> 2012-09-03 01:45:51.760376 7f8aa97f2700 10 cache get: name=.rgw+.pools.avail : > hit >> 2012-09-03 01:45:51.760381 7f8aa97f2700 20 get_obj_state: s->obj_tag was set > empty >> 2012-09-03 01:45:51.760387 7f8aa97f2700 10 moving .rgw+.pools.avail to cache > LRU end >> 2012-09-03 01:45:51.760390 7f8aa97f2700 10 cache get: name=.rgw+.pools.avail : > hit >> 2012-09-03 01:45:51.761619 7f8aa97f2700 20 rgw_create_bucket returned ret=-5 > bucket=test(@.rgw.buckets[4180.12]) >> 2012-09-03 01:45:51.761625 7f8aa97f2700 0 WARNING: set_req_state_err err_no=5 > resorting to 500 >> 2012-09-03 01:45:51.761631 7f8aa97f2700 10 --> Status: 500 >> >> 2012-09-03 01:45:51.761641 7f8aa97f2700 10 --> Content-Length: 78 >> >> 2012-09-03 01:45:51.761645 7f8aa97f2700 10 --> Accept-Ranges: bytes >> >> 2012-09-03 01:45:51.761648 7f8aa97f2700 10 --> Content-type: application/xml >> >> 2012-09-03 01:45:51.761731 7f8aa97f2700 2 req 20:0.004480:s3:PUT > /test:create_bucket:http status=500 >> 2012-09-03 01:45:51.762070 7f8aa97f2700 1 ====== req done req=0xf62550 > http_status=500 ====== >> >> >> >> >> -------- >> This e-mail may contain confidential and privileged material for the sole use > of the intended recipient. If this email is not intended for you, or you are > not responsible for the delivery of this message to the intended recipient, > please note that this message may contain SEAKR Engineering (SEAKR) > Privileged/Proprietary Information. In such a case, you are strictly > prohibited from downloading, photocopying, distributing or otherwise using > this message, its contents or attachments in any way. If you have received > this message in error, please notify us immediately by replying to this e-mail > and delete the message from your mailbox. Information contained in this > message that does not relate to the business of SEAKR is neither endorsed by > nor attributable to SEAKR. > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -------- This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR. ÿôèº{.nÇ+?·?®??+%?Ëÿ±éݶ¥?wÿº{.nÇ+?·?z?ÿuëÞ?ø§¶?¡Üš}©?²Æ zÚ&j:+v?šþø ¯ù ®w¥þ?à2?Þ?šèÚ&¢)ß¡«a¶Úÿÿûàz¿äz¹Þ?ú+?ù???Ý¢jÿ?wèþf
