Hi,
I just resolved this issue. It was probably due to a faulty region map
configuration, where more than 1 regions were marked as default. After
updating the is_master tag to false of all the non-master regions,
doing a radosgw-admin region-map update and radosgw restart, things
are working fine :)
Thanks
Abhishek
On Thu, Oct 1, 2015 at 5:21 PM, Abhishek Varshney
<abhishek.varshney@xxxxxxxxxxxx> wrote:
> Hi,
>
> I have a ceph cluster running v0.94.2 with civetweb configured on
> radosgw. I suddenly started experiencing error 500 on create_bucket.
>
> get_bucket and put/get objects are working fine.
>
> Here is my region configuration:
>
> {
> "name": "default",
> "api_name": "",
> "is_master": "false",
> "endpoints": [],
> "hostnames": [],
> "master_zone": "",
> "zones": [
> {
> "name": "default",
> "endpoints": [],
> "log_meta": "false",
> "log_data": "false",
> "bucket_index_max_shards": 0
> }
> ],
> "placement_targets": [
> {
> "name": "default-placement",
> "tags": []
> }
> ],
> "default_placement": "default-placement"
> }
>
> And this is my zone configuration:
>
> {
> "domain_root": ".rgw",
> "control_pool": ".rgw.control",
> "gc_pool": ".rgw.gc",
> "log_pool": ".log",
> "intent_log_pool": ".intent-log",
> "usage_log_pool": ".usage",
> "user_keys_pool": ".users",
> "user_email_pool": ".users.email",
> "user_swift_pool": ".users.swift",
> "user_uid_pool": ".users.uid",
> "system_key": {
> "access_key": "",
> "secret_key": ""
> },
> "placement_pools": [
> {
> "key": "default-placement",
> "val": {
> "index_pool": ".rgw.buckets.index",
> "data_pool": ".rgw.buckets",
> "data_extra_pool": ".rgw.buckets.extra"
> }
> }
> ]
> }
>
> Setting debug_rgw to 20 gives the following radosgw log in creating buckets.
>
>
> 2015-10-01 17:13:36.450824 7f6320ce7700 1 ====== starting new request
> req=0x3017890 =====
> 2015-10-01 17:13:36.450842 7f6320ce7700 2 req 27:0.000017::PUT
> /test-bucket-1/::initializing
> 2015-10-01 17:13:36.450848 7f6320ce7700 10 host=10.32.1.203:8080
> 2015-10-01 17:13:36.450850 7f6320ce7700 20 subdomain= domain= in_hosted_domain=0
> 2015-10-01 17:13:36.450890 7f6320ce7700 10 s->object=<NULL>
> s->bucket=test-bucket-1
> 2015-10-01 17:13:36.450897 7f6320ce7700 2 req 27:0.000073:s3:PUT
> /test-bucket-1/::getting op
> 2015-10-01 17:13:36.450904 7f6320ce7700 2 req 27:0.000080:s3:PUT
> /test-bucket-1/:create_bucket:authorizing
> 2015-10-01 17:13:36.450948 7f6320ce7700 10 get_canon_resource():
> dest=/test-bucket-1/
> 2015-10-01 17:13:36.450951 7f6320ce7700 10 auth_hdr:
> PUT
>
>
> Thu, 01 Oct 2015 11:43:29 GMT
> /test-bucket-1/
> 2015-10-01 17:13:36.451028 7f6320ce7700 15 calculated
> digest=o3r+2m36GnaEfWosc01+nISt+4c=
> 2015-10-01 17:13:36.451031 7f6320ce7700 15
> auth_sign=o3r+2m36GnaEfWosc01+nISt+4c=
> 2015-10-01 17:13:36.451033 7f6320ce7700 15 compare=0
> 2015-10-01 17:13:36.451043 7f6320ce7700 2 req 27:0.000219:s3:PUT
> /test-bucket-1/:create_bucket:reading permissions
> 2015-10-01 17:13:36.451049 7f6320ce7700 2 req 27:0.000225:s3:PUT
> /test-bucket-1/:create_bucket:init op
> 2015-10-01 17:13:36.451052 7f6320ce7700 2 req 27:0.000228:s3:PUT
> /test-bucket-1/:create_bucket:verifying op mask
> 2015-10-01 17:13:36.451055 7f6320ce7700 20 required_mask= 2 user.op_mask=7
> 2015-10-01 17:13:36.451057 7f6320ce7700 2 req 27:0.000233:s3:PUT
> /test-bucket-1/:create_bucket:verifying op permissions
> 2015-10-01 17:13:36.453476 7f6320ce7700 2 req 27:0.002651:s3:PUT
> /test-bucket-1/:create_bucket:verifying op params
> 2015-10-01 17:13:36.453508 7f6320ce7700 2 req 27:0.002684:s3:PUT
> /test-bucket-1/:create_bucket:executing
> 2015-10-01 17:13:36.453561 7f6320ce7700 20 get_obj_state:
> rctx=0x7f6320ce2280 obj=.rgw:test-bucket-1 state=0x302a7f0
> s->prefetch_data=0
> 2015-10-01 17:13:36.453574 7f6320ce7700 10 cache get:
> name=.rgw+test-bucket-1 : type miss (requested=22, cached=0)
> 2015-10-01 17:13:36.455132 7f6320ce7700 10 cache put: name=.rgw+test-bucket-1
> 2015-10-01 17:13:36.455150 7f6320ce7700 10 moving .rgw+test-bucket-1
> to cache LRU end
> 2015-10-01 17:13:36.455166 7f6320ce7700 0 sending create_bucket
> request to master region
> 2015-10-01 17:13:36.455169 7f6320ce7700 0 ERROR: endpoints not
> configured for upstream zone
> 2015-10-01 17:13:36.455173 7f6320ce7700 0 WARNING: set_req_state_err
> err_no=5 resorting to 500
> 2015-10-01 17:13:36.455249 7f6320ce7700 2 req 27:0.004424:s3:PUT
> /test-bucket-1/:create_bucket:http status=500
> 2015-10-01 17:13:36.455259 7f6320ce7700 1 ====== req done
> req=0x3017890 http_status=500 ======
>
>
> Any help on this would be really helpful.
>
> Thanks
> Abhishek
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
