On 5/24/19 1:15 PM, shubjero wrote:
Thanks for chiming in Konstantin! Wouldn't setting this value to 0 disable the sharding? Reference: http://docs.ceph.com/docs/mimic/radosgw/config-ref/ rgw override bucket index max shards Description:Represents the number of shards for the bucket index object, a value of zero indicates there is no sharding. It is not recommended to set a value too large (e.g. thousand) as it increases the cost for bucket listing. This variable should be set in the client or global sections so that it is automatically applied to radosgw-admin commands. Type:Integer Default:0 rgw dynamic resharding is enabled: ceph daemon mon.controller1 config show | grep rgw_dynamic_resharding "rgw_dynamic_resharding": "true", I'd like to know more about the purpose of our .usage pool and the 'usage_log_pool' in general as I cant find much about this component of ceph.
You can find docs for the usage log at http://docs.ceph.com/docs/master/radosgw/admin/#usage
Unless trimmed, the usage log will continue to grow. If you aren't using it, I'd recommend turning it off and trimming it all.
On Thu, May 23, 2019 at 11:24 PM Konstantin Shalygin <k0ste@xxxxxxxx> wrote:in the config. ``` "rgw_override_bucket_index_max_shards": "8",```. Should this be increased? Should be decreased to default `0`, I think. Modern Ceph releases resolve large omaps automatically via bucket dynamic resharding: ``` { "option": { "name": "rgw_dynamic_resharding", "type": "bool", "level": "basic", "desc": "Enable dynamic resharding", "long_desc": "If true, RGW will dynamicall increase the number of shards in buckets that have a high number of objects per shard.", "default": true, "daemon_default": "", "tags": [], "services": [ "rgw" ], "see_also": [ "rgw_max_objs_per_shard" ], "min": "", "max": "" } } ``` ``` { "option": { "name": "rgw_max_objs_per_shard", "type": "int64_t", "level": "basic", "desc": "Max objects per shard for dynamic resharding", "long_desc": "This is the max number of objects per bucket index shard that RGW will allow with dynamic resharding. RGW will trigger an automatic reshard operation on the bucket if it exceeds this number.", "default": 100000, "daemon_default": "", "tags": [], "services": [ "rgw" ], "see_also": [ "rgw_dynamic_resharding" ], "min": "", "max": "" } } ``` So when your bucket reached new 100k objects rgw will shard this bucket automatically. Some old buckets may be not sharded, like your ancients from Giant. You can check fill status like this: `radosgw-admin bucket limit check | jq '.[]'`. If some buckets is not reshared you can shart it by hand via `radosgw-admin reshard add ...`. Also, there may be some stale reshard instances (fixed ~ in 12.2.11), you can check it via `radosgw-admin reshard stale-instances list` and then remove via `reshard stale-instances rm`. k_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
