Average size is good to know, but doesn’t give much information about distribution. In a multitenant deployment, most likely there will be buckets with a variety of internal distributions too, all small, all large, mix etc. A cluster with 1M 1kB objects and 1M 10MB objects will have an average effectively the same as a cluster with 2M 5MB objects. > Do keep in mind that small objects are much more expensive then large objects. The metadata overhead becomes costly and can even become problematic if you have millions of tiny (few kb) objects. Absolutely, which is one reason why it’s good to have more detailed information, to inform architecture to consider media types, multiple storage classes, Varnish caches etc. > Hope this is still relevant... but you may want to have a look at this > example: > > https://github.com/ceph/ceph/blob/master/examples/lua/prometheus_adapter.lua > https://github.com/ceph/ceph/blob/master/examples/lua/prometheus_adapter.md > > where we log RGW object sizes to Prometheus. > would be easy to change it so it is per bucket and not per operation type. That’s really terrific, SO wish I would have had that a couple of years ago! Lua scripting is shaping up to have all kinds of great applications, especially since it’s fast. Makes me want to go write up a Grafana histogram pane, though it might be really slow over a long time region. > Just got the chance to have a look, but I see lua scripting is new in version pacific ☹ > I have octopus 15.2.14, will it be backported or no chance? I can’t speak to the potential for backporting, but running a Pacific RGW as a client of an Octopus cluster *might* work fine, but there also might be subtle incompatibilities. > > > Actually it's a good idea, metrics are already there so easy to make the Grafana dash, thank you 😊 > > Istvan Szabo > Senior Infrastructure Engineer > --------------------------------------------------- > Agoda Services Co., Ltd. > e: istvan.szabo@xxxxxxxxx > --------------------------------------------------- > > -----Original Message----- > From: Christian Rohmann <christian.rohmann@xxxxxxxxx> > Sent: Thursday, October 14, 2021 3:08 PM > To: ceph-users@xxxxxxx > Subject: Re: Metrics for object sizes > > Email received from the internet. If in doubt, don't click any link nor open any attachment ! > ________________________________ > > On 23/04/2021 03:53, Szabo, Istvan (Agoda) wrote: >> Objects inside RGW buckets like in couch base software they have their own metrics and has this information. > > Not as detailed as you would like, but how about using the bucket stats on bucket size and number of objects? > $ radosgw-admin bucket stats --bucket mybucket > > > Doing a bucket_size / number_of_objects gives you an average object size per bucket and that certainly is an indication on buckets with rather small objects. > > > > Regards > > > Christian > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |