Thanks Casey. This helped me understand the purpose of this pool. I trimmed the usage logs which reduced the number of keys stored in that index significantly and I may even disable the usage log entirely as I don't believe we use it for anything. On Fri, May 24, 2019 at 3:51 PM Casey Bodley <cbodley@xxxxxxxxxx> wrote: > > > On 5/24/19 1:15 PM, shubjero wrote: > > Thanks for chiming in Konstantin! > > > > Wouldn't setting this value to 0 disable the sharding? > > > > Reference: http://docs.ceph.com/docs/mimic/radosgw/config-ref/ > > > > rgw override bucket index max shards > > Description:Represents the number of shards for the bucket index > > object, a value of zero indicates there is no sharding. It is not > > recommended to set a value too large (e.g. thousand) as it increases > > the cost for bucket listing. This variable should be set in the client > > or global sections so that it is automatically applied to > > radosgw-admin commands. > > Type:Integer > > Default:0 > > > > rgw dynamic resharding is enabled: > > ceph daemon mon.controller1 config show | grep rgw_dynamic_resharding > > "rgw_dynamic_resharding": "true", > > > > I'd like to know more about the purpose of our .usage pool and the > > 'usage_log_pool' in general as I cant find much about this component > > of ceph. > > You can find docs for the usage log at > http://docs.ceph.com/docs/master/radosgw/admin/#usage > > Unless trimmed, the usage log will continue to grow. If you aren't using > it, I'd recommend turning it off and trimming it all. > > > > > On Thu, May 23, 2019 at 11:24 PM Konstantin Shalygin <k0ste@xxxxxxxx> wrote: > >> in the config. > >> ``` "rgw_override_bucket_index_max_shards": "8",```. Should this be > >> increased? > >> > >> Should be decreased to default `0`, I think. > >> > >> Modern Ceph releases resolve large omaps automatically via bucket dynamic resharding: > >> > >> ``` > >> > >> { > >> "option": { > >> "name": "rgw_dynamic_resharding", > >> "type": "bool", > >> "level": "basic", > >> "desc": "Enable dynamic resharding", > >> "long_desc": "If true, RGW will dynamicall increase the number of shards in buckets that have a high number of objects per shard.", > >> "default": true, > >> "daemon_default": "", > >> "tags": [], > >> "services": [ > >> "rgw" > >> ], > >> "see_also": [ > >> "rgw_max_objs_per_shard" > >> ], > >> "min": "", > >> "max": "" > >> } > >> } > >> ``` > >> > >> ``` > >> > >> { > >> "option": { > >> "name": "rgw_max_objs_per_shard", > >> "type": "int64_t", > >> "level": "basic", > >> "desc": "Max objects per shard for dynamic resharding", > >> "long_desc": "This is the max number of objects per bucket index shard that RGW will allow with dynamic resharding. RGW will trigger an automatic reshard operation on the bucket if it exceeds this number.", > >> "default": 100000, > >> "daemon_default": "", > >> "tags": [], > >> "services": [ > >> "rgw" > >> ], > >> "see_also": [ > >> "rgw_dynamic_resharding" > >> ], > >> "min": "", > >> "max": "" > >> } > >> } > >> ``` > >> > >> > >> So when your bucket reached new 100k objects rgw will shard this bucket automatically. > >> > >> Some old buckets may be not sharded, like your ancients from Giant. You can check fill status like this: `radosgw-admin bucket limit check | jq '.[]'`. If some buckets is not reshared you can shart it by hand via `radosgw-admin reshard add ...`. Also, there may be some stale reshard instances (fixed ~ in 12.2.11), you can check it via `radosgw-admin reshard stale-instances list` and then remove via `reshard stale-instances rm`. > >> > >> > >> > >> k > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |