On Thu, Feb 2, 2023 at 7:56 PM Eugen Block <eblock@xxxxxx> wrote:
> Hi,
>
> > I have a cluster with approximately one billion objects and when I run a
> PG
> > query, it shows that I have 27,000 objects per PG.
>
> which query is that, can you provide more details about that cluster and
> pool?
>
Thanks for your response
ceph df | grep mypoo
--- POOLS ---
POOL OBJECTS
mypool 1.11G
---------------
and from this, I got 8.8M objects :
for item in `radosgw-admin user list | jq -r ".[]" | head`; do
B_OBJ=$(radosgw-admin user stats --uid $item 2>/dev/null | jq -r '.stats |
select(.num_objects > 0) | .num_objects'); SUM=$((SUM + B_OBJ)); done
> However, when I run the same command per pg, the results are much
> > less, with only 20 million
> > objects being reported. For example, "rados -p <mypool> --pgid 1.xx ls |
> wc
> > -l" shows only three objects in the specified PG.
>
> It seems like your PG and object distribution might not be balanced
> very well. Did you check each PG of that pool? The individual PG's
> numbers should add up to the total number of objects. Here's a quick
> example from an almost empty pool with 8 PGs:
>
>
> storage01:~ # ceph pg ls-by-pool volumes | awk '{print $1" - "$2}'
> PG - OBJECTS
> 3.0 - 2
> 3.1 - 0
> 3.2 - 2
> 3.3 - 1
> 3.4 - 3
> 3.5 - 0
> 3.6 - 2
> 3.7 - 0
>
>
```
~# ceph pg ls-by-pool mypool | awk '{print $1" - "$2}
...
9.ff2 - 271268
9.ff3 - 271046
9.ff4 - 271944
9.ff5 - 270864
9.ff6 - 272122
9.ff7 - 272244
9.ff8 - 271638
9.ff9 - 271702
9.ffa - 270906
9.ffb - 271114
9.ffc - 271986
9.ffd - 270766
9.ffe - 271702
9.fff - 271693
...
```
> and it sums up to 10 objects which matches the total stats:
>
> storage01:~ # ceph df | grep -E "OBJECTS|volumes"
> POOL ID PGS STORED OBJECTS USED %USED
> MAX AVAIL
> volumes 3 8 379 B 10 960 KiB 0 7.5 GiB
>
> Regards,
> Eugen
>
> Zitat von Ramin Najjarbashi <ramin.najarbashi@xxxxxxxxx>:
>
> > Hi
> > I hope this email finds you well. I am reaching out to you because I have
> > encountered an issue with my CEPH Bluestore cluster and I am seeking your
> > assistance.
> > I have a cluster with approximately one billion objects and when I run a
> PG
> > query, it shows that I have 27,000 objects per PG.
> > I have run the following command: "rados -p <mypool> ls | wc -l" which
> > returns the correct number of one billion objects. However, when I run
> the
> > same command per pg, the results are much less, with only 20 million
> > objects being reported. For example, "rados -p <mypool> --pgid 1.xx ls |
> wc
> > -l" shows only three objects in the specified PG.
> > This is a significant discrepancy and I am concerned about the integrity
> of
> > my data.
> > Do you have any idea about this discrepancy?
> >
> > p.s:
> > I have a total of 30 million objects in a single bucket and versioning
> has
> > not been enabled for this particular bucket.
> >
> > Thank you for your time and I look forward to your response.
> >
> > Best regards,
> > Ramin
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
