Hi Drew, On Friday, March 5th, 2021 at 20:22, Drew Weaver <drew.weaver@xxxxxxxxxx> wrote: > Sorry for multi-reply, I got that command to run: > > for obj in $(rados -p default.rgw.buckets.index ls | grep 2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637); do printf "%-60s %7d\n" $obj $(rados -p default.rgw.buckets.index listomapkeys $obj | wc -l); done; > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.0 10423 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.15 10445 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.3 10542 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.6 10511 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.13 10414 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.12 10479 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.2 10486 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.5 10448 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.4 10470 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.8 10474 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.1 10470 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.10 10411 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.7 10445 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.14 10413 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.9 10356 > .dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.637.11 10410 So this looks fine, it means that this bucket is sharded properly and the index keys are evenly distributed. However, the problematic object is `.dir.2b67ef7c-2015-4ca0-bf50-b7595d01e46e.74194.213.0`; you said there's no bucket with that ID, which means it's probably a stale instance. Does it still exist in `default.rgw.buckets.index`? If so, you can try running `radosgw-admin reshard stale-instances rm`, it should get rid of it (you can verify afterwards with `rados -p default.rgw.buckets.index ls`). -- Ben _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |