Hi, I have a ceph cluster running v0.94.2 with civetweb configured on radosgw. I suddenly started experiencing error 500 on create_bucket. get_bucket and put/get objects are working fine. Here is my region configuration: { "name": "default", "api_name": "", "is_master": "false", "endpoints": [], "hostnames": [], "master_zone": "", "zones": [ { "name": "default", "endpoints": [], "log_meta": "false", "log_data": "false", "bucket_index_max_shards": 0 } ], "placement_targets": [ { "name": "default-placement", "tags": [] } ], "default_placement": "default-placement" } And this is my zone configuration: { "domain_root": ".rgw", "control_pool": ".rgw.control", "gc_pool": ".rgw.gc", "log_pool": ".log", "intent_log_pool": ".intent-log", "usage_log_pool": ".usage", "user_keys_pool": ".users", "user_email_pool": ".users.email", "user_swift_pool": ".users.swift", "user_uid_pool": ".users.uid", "system_key": { "access_key": "", "secret_key": "" }, "placement_pools": [ { "key": "default-placement", "val": { "index_pool": ".rgw.buckets.index", "data_pool": ".rgw.buckets", "data_extra_pool": ".rgw.buckets.extra" } } ] } Setting debug_rgw to 20 gives the following radosgw log in creating buckets. 2015-10-01 17:13:36.450824 7f6320ce7700 1 ====== starting new request req=0x3017890 ===== 2015-10-01 17:13:36.450842 7f6320ce7700 2 req 27:0.000017::PUT /test-bucket-1/::initializing 2015-10-01 17:13:36.450848 7f6320ce7700 10 host=10.32.1.203:8080 2015-10-01 17:13:36.450850 7f6320ce7700 20 subdomain= domain= in_hosted_domain=0 2015-10-01 17:13:36.450890 7f6320ce7700 10 s->object=<NULL> s->bucket=test-bucket-1 2015-10-01 17:13:36.450897 7f6320ce7700 2 req 27:0.000073:s3:PUT /test-bucket-1/::getting op 2015-10-01 17:13:36.450904 7f6320ce7700 2 req 27:0.000080:s3:PUT /test-bucket-1/:create_bucket:authorizing 2015-10-01 17:13:36.450948 7f6320ce7700 10 get_canon_resource(): dest=/test-bucket-1/ 2015-10-01 17:13:36.450951 7f6320ce7700 10 auth_hdr: PUT Thu, 01 Oct 2015 11:43:29 GMT /test-bucket-1/ 2015-10-01 17:13:36.451028 7f6320ce7700 15 calculated digest=o3r+2m36GnaEfWosc01+nISt+4c= 2015-10-01 17:13:36.451031 7f6320ce7700 15 auth_sign=o3r+2m36GnaEfWosc01+nISt+4c= 2015-10-01 17:13:36.451033 7f6320ce7700 15 compare=0 2015-10-01 17:13:36.451043 7f6320ce7700 2 req 27:0.000219:s3:PUT /test-bucket-1/:create_bucket:reading permissions 2015-10-01 17:13:36.451049 7f6320ce7700 2 req 27:0.000225:s3:PUT /test-bucket-1/:create_bucket:init op 2015-10-01 17:13:36.451052 7f6320ce7700 2 req 27:0.000228:s3:PUT /test-bucket-1/:create_bucket:verifying op mask 2015-10-01 17:13:36.451055 7f6320ce7700 20 required_mask= 2 user.op_mask=7 2015-10-01 17:13:36.451057 7f6320ce7700 2 req 27:0.000233:s3:PUT /test-bucket-1/:create_bucket:verifying op permissions 2015-10-01 17:13:36.453476 7f6320ce7700 2 req 27:0.002651:s3:PUT /test-bucket-1/:create_bucket:verifying op params 2015-10-01 17:13:36.453508 7f6320ce7700 2 req 27:0.002684:s3:PUT /test-bucket-1/:create_bucket:executing 2015-10-01 17:13:36.453561 7f6320ce7700 20 get_obj_state: rctx=0x7f6320ce2280 obj=.rgw:test-bucket-1 state=0x302a7f0 s->prefetch_data=0 2015-10-01 17:13:36.453574 7f6320ce7700 10 cache get: name=.rgw+test-bucket-1 : type miss (requested=22, cached=0) 2015-10-01 17:13:36.455132 7f6320ce7700 10 cache put: name=.rgw+test-bucket-1 2015-10-01 17:13:36.455150 7f6320ce7700 10 moving .rgw+test-bucket-1 to cache LRU end 2015-10-01 17:13:36.455166 7f6320ce7700 0 sending create_bucket request to master region 2015-10-01 17:13:36.455169 7f6320ce7700 0 ERROR: endpoints not configured for upstream zone 2015-10-01 17:13:36.455173 7f6320ce7700 0 WARNING: set_req_state_err err_no=5 resorting to 500 2015-10-01 17:13:36.455249 7f6320ce7700 2 req 27:0.004424:s3:PUT /test-bucket-1/:create_bucket:http status=500 2015-10-01 17:13:36.455259 7f6320ce7700 1 ====== req done req=0x3017890 http_status=500 ====== Any help on this would be really helpful. Thanks Abhishek _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com