Hi,
--
MSI has an erasure coded ceph pool accessible by the radosgw interface. We recently upgraded to Jewel from Hammer. Several days ago, we experienced issues with a couple of the rados gateway servers and inadvertently deployed older Hammer versions of the radosgw instances. This configuration was running for a couple of days. We removed the Hammer versions and re-deployed the Jewel versions of the radosgw. S3cmd querying of some of the buckets are now reporting 'NoSuchKey'. This seems to have started when the Hammer versions were deployed and this is now persistent. While radosgw-admin seems to know about the bucket, checks on the buckets itself now fail:
# radosgw-admin bucket list --uid=jmcdonal
[
"bigbucket",
"hpmesabiinfo",
"jmarchive",
"jmcdon",
"jmcdonal",
"jmcdontest3",
"jmtestbigfiles",
"laptopbackup",
"mesabihpsite",
"msisoftware",
"tcga"
]
# radosgw-admin bucket check --check-head-obj-locator --bucket=jmarchive
ERROR: store->list_objects(): (2) No such file or directory
It seems like some metadata about the buckets or objects has been lost, is there a way to recover the bucket?
I searched through the archives but didn't find anything exactly like this. Please point me to the documentation if this has been seen before.
Thanks,
Jeff
Jeffrey McDonald, PhD Assistant Director for HPC Operations Minnesota Supercomputing Institute University of Minnesota Twin Cities 599 Walter Library email: jeffrey.mcdonald@xxxxxxxxxxx 117 Pleasant St SE phone: +1 612 625-6905 Minneapolis, MN 55455 fax: +1 612 624-8861
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
