Hi Eugen, Thanks for the suggestion. The object counts of rbd pool are still stay on 430.11K. (all images were deleted 3 days +.) I will keep monitor it and post the results here. Regs, Icy On Wed, 20 May 2020 at 15:12, Eugen Block <eblock@xxxxxx> wrote: > The rbd_info, rbd_directory objects will remain until you delete the > pool, you don't need to clean that up, e.g. if you decide to create > new rbd images in there. > The number of remaining objects usually slowly decreases depending on > the amount of data that was deleted. Just last week I deleted a 2 TB > rbd image from a pool and it took two days until the number of objects > cleaned up, so it's nothing unusual. Just watch the number from time > to time and this thread in case the numbers don't decrease. > > > Zitat von icy chan <icy.kf.chan@xxxxxxxxx>: > > > Hi Eugen, > > > > Thanks for your reply. > > > > The problem is all rbd images were removed from pool rbd days ago. i.e. > > both below commands also return empty: > > $ rbd ls rbd > > $ rados -p rbd listomapkeys rbd_directory > > > > But below rados df still located 430K objects. Any other methods can I > dig > > out those ghost objects? > > > > $ rados -p rbd df > > POOL_NAME USED OBJECTS CLONES COPIES MISSING_ON_PRIMARY UNFOUND > DEGRADED > > RD_OPS RD WR_OPS WR USED COMPR UNDER COMPR > > rbd 18 MiB 430107 0 1290321 0 0 > 0 > > 141454780 6.9 TiB 42395431 11 TiB 0 B 0 B > > > > $ rados -p rbd ls | while read obj; do echo "- "$obj; rados -p rbd stat > > $obj; rados -p rbd listomapkeys $obj; echo ; done > > - gateway.conf > > rbd/gateway.conf mtime 2020-05-18 11:15:30.000000, size 8869 > > > > - rbd_directory > > rbd/rbd_directory mtime 2020-05-18 10:59:50.000000, size 0 > > > > - rbd_info > > rbd/rbd_info mtime 2020-04-16 15:07:42.000000, size 19 > > > > - rbd_trash > > rbd/rbd_trash mtime 2020-05-18 11:15:30.000000, size 0 > > > > > > Regs, > > Icy > > > > > > > > On Tue, 19 May 2020 at 14:57, Eugen Block <eblock@xxxxxx> wrote: > > > >> That's not wrong, those are expected objects that contain information > >> about your rbd images. If you take a look into the rbd_directory > >> (while you have images in there) you'll find something like this: > >> > >> host:~ $ rados -p pool listomapkeys rbd_directory > >> > >> id_fe976bcfb968bf > >> id_ffc37728edbdab > >> name_01673d5d-4b12-4a44-8793-403581f7d808_disk > >> name_01673d5d-4b12-4a44-8793-403581f7d808_disk.config > >> name_volume-8a1a0825-1163-44bc-abe2-1a711daea07b > >> > >> > >> The rbd ls command uses the rbd_directory object to read from, an > >> excerpt from rbd man page: > >> > >> ---snip--- > >> ls [-l | –long] [pool-name] > >> > >> Will list all rbd images listed in the rbd_directory object. > >> ---snip--- > >> > >> > >> The gateway.conf is your iSCSI gateway configuration stored in the > cluster. > >> > >> > >> Zitat von icy chan <icy.kf.chan@xxxxxxxxx>: > >> > >> > Hi, > >> > > >> > The numbers of object counts from "rados df" and "rados ls" are > different > >> > in my testing environment. I think it maybe some zero bytes or unclean > >> > objects since I removed all rbd images on top of it few days ago. > >> > How can I make it right / found out where are those ghost objects? Or > i > >> > should ignore it since the numbers was not that high. > >> > > >> > $ rados -p rbd df > >> > POOL_NAME USED OBJECTS CLONES COPIES MISSING_ON_PRIMARY UNFOUND > >> DEGRADED > >> > RD_OPS RD WR_OPS WR USED COMPR UNDER COMPR > >> > rbd 18 MiB 430107 0 1290321 0 0 > >> 0 > >> > 141243877 6.9 TiB 42395431 11 TiB 0 B 0 B > >> > > >> > $ rados -p rbd ls | wc -l > >> > 4 > >> > > >> > $ rados -p rbd ls > >> > gateway.conf > >> > rbd_directory > >> > rbd_info > >> > rbd_trash > >> > > >> > Regs, > >> > Icy > >> > _______________________________________________ > >> > ceph-users mailing list -- ceph-users@xxxxxxx > >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx > >> > >> > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@xxxxxxx > >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > >> > > > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |