Hi > Check the S3 Bucket OPS at : http://ceph.com/docs/master/radosgw/s3/bucketops/ I've read that as well, but I'm having other issues getting an App to run against our Ceph S3 GW, maybe you have a few hints on this as well... Got the cluster working for rbd+cephFS and have initial verified the S3 service as working. But have two-three issues currently: 1. when the App initial launches it want to create a bucket to hold App data, seems the bucket gets created, only the App (which is known to work against AWS + Scality S3) doesn't recognize the response so it never gets to run. Got a tcp dump of: Request: PUT /mssCl/ HTTP/1.1 Host: rgw.gsp Authorization: AWS <auth id> Date: Fri, 06 Mar 2015 10:41:14 GMT Content-Length: 0 Response: HTTP/1.1 200 OK Date: Fri, 06 Mar 2015 10:41:14 GMT Server: Apache/2.2.22 (Fedora) Connection: close Transfer-Encoding: chunked Content-Type: application/xml This response makes the App say: S3.createBucket, class <S3>, code <UnexpectedContent>, message <Inconsistency in S3 response. error response is not a valid xml message> Are our S3 GW not responding properly? 2. at every create bucket OP the GW create what looks like new containers for ACLs in .rgw pool, is this normal or howto avoid such multiple objects clottering the GW pools? # rados -p .rgw ls .bucket.meta.mssCl:default.6309817.1 .bucket.meta.mssCl:default.6187712.3 .bucket.meta.mssCl:default.6299841.7 .bucket.meta.mssCl:default.6309817.5 .bucket.meta.mssCl:default.6187712.2 .bucket.meta.mssCl:default.6187712.19 .bucket.meta.mssCl:default.6187712.12 mssCl ... # rados -p .rgw listxattr .bucket.meta.mssCl:default.6187712.12 ceph.objclass.version user.rgw.acl the GW log shows: 2015-03-06 09:16:04.505655 7fd3ac5e6700 1 ====== starting new request req=0x7fd3d8009c40 ===== 2015-03-06 09:16:04.506131 7fd3ac5e6700 2 req 2:0.000484::PUT /mssCl/::initializing 2015-03-06 09:16:04.507110 7fd3ac5e6700 10 host=rgw.gsp rgw_dns_name=rgw.gsp.sprawl.dk 2015-03-06 09:16:04.510138 7fd3ac5e6700 10 s->object=<NULL> s->bucket=mssCl 2015-03-06 09:16:04.511087 7fd3ac5e6700 2 req 2:0.005439:s3:PUT /mssCl/::getting op 2015-03-06 09:16:04.512041 7fd3ac5e6700 2 req 2:0.006395:s3:PUT /mssCl/:create_bucket:authorizing 2015-03-06 09:16:04.515173 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc007600 obj=.users:WL4EJJYTLVYXEHNR6QSA state=0x7fd3dc00c5d8 s->pre fetch_data=0 2015-03-06 09:16:04.516185 7fd3ac5e6700 10 cache get: name=.users+WL4EJJYTLVYXEHNR6QSA : type miss (requested=6, cached=3) 2015-03-06 09:16:04.542851 7fd3ac5e6700 10 cache put: name=.users+WL4EJJYTLVYXEHNR6QSA 2015-03-06 09:16:04.542933 7fd3ac5e6700 10 moving .users+WL4EJJYTLVYXEHNR6QSA to cache LRU end 2015-03-06 09:16:04.544187 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.546138 7fd3ac5e6700 10 cache get: name=.users+WL4EJJYTLVYXEHNR6QSA : hit 2015-03-06 09:16:04.550060 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc007600 obj=.users.uid:mx9mss state=0x7fd3dc00daa8 s->prefetch_data =0 2015-03-06 09:16:04.551777 7fd3ac5e6700 10 cache get: name=.users.uid+mx9mss : hit 2015-03-06 09:16:04.551927 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.554924 7fd3ac5e6700 10 cache get: name=.users.uid+mx9mss : hit 2015-03-06 09:16:04.559059 7fd3ac5e6700 10 chain_cache_entry: cache_locator=.users.uid+mx9mss 2015-03-06 09:16:04.565872 7fd3ac5e6700 10 get_canon_resource(): dest=/mssCl/ 2015-03-06 09:16:04.566637 7fd3ac5e6700 10 auth_hdr: PUT Fri, 06 Mar 2015 08:16:03 GMT /mssCl/ 2015-03-06 09:16:04.600259 7fd3ac5e6700 15 calculated digest=7nava2kuDurTiUVqxpn8OkP5I10= 2015-03-06 09:16:04.600529 7fd3ac5e6700 15 auth_sign=7nava2kuDurTiUVqxpn8OkP5I10= 2015-03-06 09:16:04.600563 7fd3ac5e6700 15 compare=0 2015-03-06 09:16:04.600653 7fd3ac5e6700 2 req 2:0.095001:s3:PUT /mssCl/:create_bucket:reading permissions 2015-03-06 09:16:04.600779 7fd3ac5e6700 2 req 2:0.095133:s3:PUT /mssCl/:create_bucket:init op 2015-03-06 09:16:04.600836 7fd3ac5e6700 2 req 2:0.095191:s3:PUT /mssCl/:create_bucket:verifying op mask 2015-03-06 09:16:04.600888 7fd3ac5e6700 20 required_mask= 2 user.op_mask=7 2015-03-06 09:16:04.600925 7fd3ac5e6700 2 req 2:0.095281:s3:PUT /mssCl/:create_bucket:verifying op permissions 2015-03-06 09:16:04.643915 7fd3ac5e6700 2 req 2:0.138261:s3:PUT /mssCl/:create_bucket:verifying op params 2015-03-06 09:16:04.644421 7fd3ac5e6700 2 req 2:0.138775:s3:PUT /mssCl/:create_bucket:executing 2015-03-06 09:16:04.644912 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3ac5e55d0 obj=.rgw:mssCl state=0x7fd3dc00ebe8 s->prefetch_data=0 2015-03-06 09:16:04.645328 7fd3ac5e6700 10 cache get: name=.rgw+mssCl : type miss (requested=6, cached=19) 2015-03-06 09:16:04.665683 7fd3ac5e6700 10 cache put: name=.rgw+mssCl 2015-03-06 09:16:04.665844 7fd3ac5e6700 10 moving .rgw+mssCl to cache LRU end 2015-03-06 09:16:04.666369 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.666437 7fd3ac5e6700 0 WARNING: couldn't find acl header for object, generating default 2015-03-06 09:16:04.666670 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc00cd10 obj=.users.uid:mx9mss state=0x7fd3dc00f438 s->prefetch_data =0 2015-03-06 09:16:04.666826 7fd3ac5e6700 10 cache get: name=.users.uid+mx9mss : hit 2015-03-06 09:16:04.666905 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.667412 7fd3ac5e6700 10 cache get: name=.users.uid+mx9mss : hit 2015-03-06 09:16:04.714453 7fd3ac5e6700 10 cache put: name=.rgw+.bucket.meta.mssCl:default.6306353.2 2015-03-06 09:16:04.714612 7fd3ac5e6700 10 adding .rgw+.bucket.meta.mssCl:default.6306353.2 to cache LRU end 2015-03-06 09:16:04.714663 7fd3ac5e6700 10 updating xattr: name=user.rgw.acl bl.length()=183 2015-03-06 09:16:04.715288 7fd3ac5e6700 10 distributing notification oid=notify.2 bl.length()=664 2015-03-06 09:16:04.728763 7fd3f8dfa700 10 RGWWatcher::notify() opcode=1 ver=391 bl.length()=664 2015-03-06 09:16:04.729383 7fd3f8dfa700 10 cache put: name=.rgw+.bucket.meta.mssCl:default.6306353.2 2015-03-06 09:16:04.729434 7fd3f8dfa700 10 moving .rgw+.bucket.meta.mssCl:default.6306353.2 to cache LRU end 2015-03-06 09:16:04.729495 7fd3f8dfa700 10 updating xattr: name=user.rgw.acl bl.length()=183 2015-03-06 09:16:04.762798 7fd3ac5e6700 10 removing .rgw+mssCl from cache 2015-03-06 09:16:04.763660 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc00f750 obj=.rgw:mssCl state=0x7fd3dc0189e8 s->prefetch_data=0 2015-03-06 09:16:04.763797 7fd3ac5e6700 10 cache get: name=.rgw+mssCl : miss 2015-03-06 09:16:04.774689 7fd3ac5e6700 10 cache put: name=.rgw+mssCl 2015-03-06 09:16:04.774830 7fd3ac5e6700 10 adding .rgw+mssCl to cache LRU end 2015-03-06 09:16:04.774919 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.775376 7fd3ac5e6700 10 cache get: name=.rgw+mssCl : type miss (requested=17, cached=22) 2015-03-06 09:16:04.775533 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc00f750 obj=.rgw:mssCl state=0x7fd3dc0189e8 s->prefetch_data=0 2015-03-06 09:16:04.775627 7fd3ac5e6700 10 cache get: name=.rgw+mssCl : hit 2015-03-06 09:16:04.775689 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.775808 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc00f750 obj=.rgw:mssCl state=0x7fd3dc0189e8 s->prefetch_data=0 2015-03-06 09:16:04.775848 7fd3ac5e6700 20 state for obj=.rgw:mssCl is not atomic, not appending atomic test 2015-03-06 09:16:04.776479 7fd3ac5e6700 20 rados->read obj-ofs=0 read_ofs=0 read_len=524288 2015-03-06 09:16:04.788552 7fd3ac5e6700 20 rados->read r=0 bl.length=142 2015-03-06 09:16:04.788789 7fd3ac5e6700 10 cache put: name=.rgw+mssCl 2015-03-06 09:16:04.788829 7fd3ac5e6700 10 moving .rgw+mssCl to cache LRU end 2015-03-06 09:16:04.789373 7fd3ac5e6700 20 rgw_get_bucket_info: bucket instance: mssCl(@{i=.rgw.buckets.index,e=.rgw.buckets.extra}.rgw.b uckets[default.6187712.1]) 2015-03-06 09:16:04.789497 7fd3ac5e6700 20 reading from .rgw:.bucket.meta.mssCl:default.6187712.1 2015-03-06 09:16:04.789689 7fd3ac5e6700 20 get_obj_state: rctx=0x7fd3dc018b10 obj=.rgw:.bucket.meta.mssCl:default.6187712.1 state=0x7fd3d c0160d8 s->prefetch_data=0 2015-03-06 09:16:04.789818 7fd3ac5e6700 10 cache get: name=.rgw+.bucket.meta.mssCl:default.6187712.1 : hit 2015-03-06 09:16:04.790289 7fd3ac5e6700 20 get_obj_state: s->obj_tag was set empty 2015-03-06 09:16:04.790343 7fd3ac5e6700 20 Read xattr: user.rgw.acl 2015-03-06 09:16:04.790368 7fd3ac5e6700 20 Read xattr: user.rgw.idtag 2015-03-06 09:16:04.790385 7fd3ac5e6700 20 Read xattr: user.rgw.manifest 2015-03-06 09:16:04.790459 7fd3ac5e6700 10 cache get: name=.rgw+.bucket.meta.mssCl:default.6187712.1 : hit 2015-03-06 09:16:04.790664 7fd3ac5e6700 10 chain_cache_entry: cache_locator=.rgw+mssCl 2015-03-06 09:16:04.790702 7fd3ac5e6700 10 chain_cache_entry: cache_locator=.rgw+.bucket.meta.mssCl:default.6187712.1 2015-03-06 09:16:04.791327 7fd3ac5e6700 10 removing .rgw+.bucket.meta.mssCl:default.6306353.2 from cache 2015-03-06 09:16:04.791574 7fd3ac5e6700 10 distributing notification oid=notify.2 bl.length()=236 2015-03-06 09:16:04.808884 7fd3f8dfa700 10 RGWWatcher::notify() opcode=1 ver=391 bl.length()=236 2015-03-06 09:16:04.850695 7fd3ac5e6700 20 rgw_create_bucket returned ret=-17 bucket=mssCl(@{i=.rgw.buckets.index,e=.rgw.buckets.extra}.r gw.buckets[default.6306353.2]) 2015-03-06 09:16:04.870353 7fd3ac5e6700 2 req 2:0.364701:s3:PUT /mssCl/:create_bucket:http status=200 2015-03-06 09:16:04.870521 7fd3ac5e6700 1 ====== req done req=0x7fd3d8009c40 http_status=200 ====== 3. What are BCP for maintaining GW pools, need I run something like GC / cleanup OPs / log object pruning etc. any pointers to doc here for? /Steffen
Attachment:
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
