Re: RGW: ERROR: failed to distribute cache

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



BTW, I have 10 RGW load balanced through Apache.

When restarting one of them I get the following messages in log:
2016-06-14 14:44:15.919801 7fd4728dea40  2 all 8 watchers are set,
enabling cache
2016-06-14 14:44:15.919879 7fce370f7700  2 garbage collection: start
2016-06-14 14:44:15.919990 7fce368f6700  2 object expiration: start
2016-06-14 14:44:15.920534 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.15
2016-06-14 14:44:15.921257 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.16
2016-06-14 14:44:15.922145 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.17
2016-06-14 14:44:15.923772 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.18
2016-06-14 14:44:15.924557 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.19
2016-06-14 14:44:15.925400 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.20
2016-06-14 14:44:15.926349 7fd4728dea40  0 starting handler: fastcgi
2016-06-14 14:44:15.927125 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.21
2016-06-14 14:44:15.927897 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.22
2016-06-14 14:44:15.928412 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.23
2016-06-14 14:44:15.929042 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.24
2016-06-14 14:44:15.930752 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.25
2016-06-14 14:44:15.931313 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.26
2016-06-14 14:44:15.932482 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.27
2016-06-14 14:44:15.933237 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.28
2016-06-14 14:44:15.934097 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.29
2016-06-14 14:44:15.934660 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.30
2016-06-14 14:44:15.936322 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.31
2016-06-14 14:44:15.936979 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.0
2016-06-14 14:44:15.937559 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.1
2016-06-14 14:44:15.938222 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.2
2016-06-14 14:44:15.939000 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.3
2016-06-14 14:44:15.939622 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.4
2016-06-14 14:44:15.940135 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.5
2016-06-14 14:44:15.940669 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.6
2016-06-14 14:44:15.941227 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.7
2016-06-14 14:44:15.941854 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.8
2016-06-14 14:44:15.942333 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.9
2016-06-14 14:44:15.943036 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.10
2016-06-14 14:44:15.944708 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.11
2016-06-14 14:44:15.946347 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.12
2016-06-14 14:44:15.947001 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.13
2016-06-14 14:44:15.947610 7fce370f7700  0 RGWGC::process() failed to
acquire lock on gc.14
2016-06-14 14:44:15.947615 7fce370f7700  2 garbage collection: stop
2016-06-14 14:44:15.947949 7fd4728dea40 -1 rgw realm watcher: Failed
to watch realms.87abf44e-cab3-48c4-b012-0a9247519a5b.control with (2)
No such file or directory
2016-06-14 14:44:15.948370 7fd4728dea40 -1 rgw realm watcher: Failed
to establish a watch on RGWRealm, disabling dynamic reconfiguration.

2016-06-14 17:34 GMT+03:00 Василий Ангапов <angapov@xxxxxxxxx>:
> I also get the following:
>
> $ radosgw-admin period update --commit
> 2016-06-14 14:32:28.982847 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:periods.87abf44e-cab3-48c4-b012-0a9247519a5b:staging
> 2016-06-14 14:32:38.991846 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:periods.87abf44e-cab3-48c4-b012-0a9247519a5b:staging.latest_epoch
> 2016-06-14 14:32:49.002380 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:periods.af0b6743-82ba-4517-bd51-36bdfbe48f9f.3
> 2016-06-14 14:32:59.013307 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:periods.af0b6743-82ba-4517-bd51-36bdfbe48f9f.latest_epoch
> 2016-06-14 14:33:09.023554 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:periods.af0b6743-82ba-4517-bd51-36bdfbe48f9f.latest_epoch
> 2016-06-14 14:33:19.034593 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:zonegroup_info.bef0aa4e-6670-4c39-8520-ee51140424cc
> 2016-06-14 14:33:29.043825 7fed392baa40  0 ERROR: failed to distribute
> cache for .rgw.root:zonegroups_names.ed
> 2016-06-14 14:33:29.046386 7fed392baa40  0 Realm notify failed with -2
> {
>     "id": "af0b6743-82ba-4517-bd51-36bdfbe48f9f",
>     "epoch": 3,
>     "predecessor_uuid": "f2645d83-b1b4-4045-bf26-2b762c71937b",
>     "sync_status": [
>         "",
>         "",
>
> 2016-06-14 17:12 GMT+03:00 Василий Ангапов <angapov@xxxxxxxxx>:
>> Hello,
>>
>> I have Ceph 10.2.1 and when creating user in RGW I get the following error:
>>
>> $ radosgw-admin user create --uid=test --display-name="test"
>> 2016-06-14 14:07:32.332288 7f00a4487a40  0 ERROR: failed to distribute
>> cache for ed-1.rgw.meta:.meta:user:test:_dW3fzQ3UX222SWQvr3qeHYR:1
>> 2016-06-14 14:07:42.338251 7f00a4487a40  0 ERROR: failed to distribute
>> cache for ed-1.rgw.users.uid:test
>> 2016-06-14 14:07:52.362768 7f00a4487a40  0 ERROR: failed to distribute
>> cache for ed-1.rgw.users.keys:3J7DOREPC0ZLVFTMIW75
>> {
>>     "user_id": "test",
>>     "display_name": "test",
>>     "email": "",
>>     "suspended": 0,
>>     "max_buckets": 1000,
>>     "auid": 0,
>>     "subusers": [],
>>     "keys": [
>>         {
>>             "user": "melesta",
>>             "access_key": "***",
>>             "secret_key": "***"
>>         }
>>     ],
>>     "swift_keys": [],
>>     "caps": [],
>>     "op_mask": "read, write, delete",
>>     "default_placement": "",
>>     "placement_tags": [],
>>     "bucket_quota": {
>>         "enabled": false,
>>         "max_size_kb": -1,
>>         "max_objects": -1
>>     },
>>     "user_quota": {
>>         "enabled": false,
>>         "max_size_kb": -1,
>>         "max_objects": -1
>>     },
>>     "temp_url_keys": []
>> }
>>
>> What does it mean? Is something wrong?
>>
>> Thanks!
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux