Re: 10x more used space than expected

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Aaaah many many thanks for the information rich! It helped a lot!

Indeed the rados operations documentation lack a bunch of advanced commands
explanations.

I’ve finally find the exact and explicit command explanation in here for
the record: https://docs.ceph.com/en/latest/radosgw/layout/#metadata

Regarding my own metadata get bucket:
https://paste.opendev.org/show/bOI6gItMdDMvXSs2Swf5/

&

For the metadata get bucket.instance:
https://paste.opendev.org/show/bRPec1KViuklmNIjasU4/

I’ve also checked our min_alloc size, I’m using default 64K with EC of 2+1
which seems to match the data space amplification spreadsheet as a bunch of
our registry objects are 0 to 1780 octets size and ranging from there to
large object that fit the min_alloc size.

Here is an object detailed:
https://paste.opendev.org/show/bp7nvVPq5a9mbF6Ad3Jc/

So all in all we may well be in that case.

I still have a question about that, is there a way to optimize that a bit
as x10 is a bit huge even if we get way more available space, I’m wondering
if the fragmented free space could be retrieved or at least ceph count for
the real object size used and not the reserved size?

If anyone is having thoughts on the situation in here I’m all hears out if
it can improve and reduce the storage size usage.

Kind regards,
G.

Le mar. 14 mars 2023 à 21:44, Richard Bade <hitrich@xxxxxxxxx> a écrit :

> Hi,
> I found the documentation for metadata get to be unhelpful for what
> syntax to use. I eventually found that it's this:
> radosgw-admin metadata get bucket:{bucket_name}
> or
> radosgw-admin metadata get bucket.instance:{bucket_name}:{instance_id}
>
> Hopefully that helps you or someone else struggling with this.
>
> Rich
>
> On Wed, 15 Mar 2023 at 07:18, Gaël THEROND <gael.therond@xxxxxxxxxxxx>
> wrote:
> >
> > Alright,
> > Seems something is odd out there, if I do a radosgw-admin metadata list
> >
> > I’ve got the following list:
> >
> > [
> >     ”bucket”,
> >     ”bucket.instance”,
> >     ”otp”,
> >     ”user”
> > ]
> >
> > BUT
> >
> > When I try a radosgw-admin metadata get bucket or bucket.instance it
> > complain with the following error:
> >
> > ERROR: can’t get key: (22) Invalid argument
> >
> > Ok, fine for the api, I’ll deal with the s3 api.
> >
> > Even if a radosgw-admin bucket flush version —keep-current or something
> > similar would be much appreciated xD
> >
> > Le mar. 14 mars 2023 à 19:07, Robin H. Johnson <robbat2@xxxxxxxxxx> a
> > écrit :
> >
> > > On Tue, Mar 14, 2023 at 06:59:51PM +0100, Gaël THEROND wrote:
> > > > Versioning wasn’t enabled, at least not explicitly and for the
> > > > documentation it isn’t enabled by default.
> > > >
> > > > Using nautilus.
> > > >
> > > > I’ll get all the required missing information on tomorrow morning,
> thanks
> > > > for the help!
> > > >
> > > > Is there a way to tell CEPH to delete versions that aren’t current
> used
> > > one
> > > > with radosgw-admin?
> > > >
> > > > If not I’ll use the rest api no worries.
> > > Nope, s3 API only.
> > >
> > > You should also check for incomplete multiparts. For that, I recommend
> > > using AWSCLI or boto directly. Specifically not s3cmd, because s3cmd
> > > doesn't respect the <IsTruncated> flag properly.
> > >
> > > --
> > > Robin Hugh Johnson
> > > Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
> > > E-Mail   : robbat2@xxxxxxxxxx
> > > GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
> > > GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
> > >
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux