On Fri, Nov 18, 2016 at 1:14 PM, Jeffrey McDonald <jmcdonal@xxxxxxx> wrote: > Hi, > > MSI has an erasure coded ceph pool accessible by the radosgw interface. > We recently upgraded to Jewel from Hammer. Several days ago, we > experienced issues with a couple of the rados gateway servers and > inadvertently deployed older Hammer versions of the radosgw instances. > This configuration was running for a couple of days. We removed the Hammer > versions and re-deployed the Jewel versions of the radosgw. S3cmd > querying of some of the buckets are now reporting 'NoSuchKey'. This seems > to have started when the Hammer versions were deployed and this is now > persistent. While radosgw-admin seems to know about the bucket, checks on > the buckets itself now fail: > > # radosgw-admin bucket list --uid=jmcdonal > [ > "bigbucket", > "hpmesabiinfo", > "jmarchive", > "jmcdon", > "jmcdonal", > "jmcdontest3", > "jmtestbigfiles", > "laptopbackup", > "mesabihpsite", > "msisoftware", > "tcga" > ] > # radosgw-admin bucket check --check-head-obj-locator --bucket=jmarchive > ERROR: store->list_objects(): (2) No such file or directory > > It seems like some metadata about the buckets or objects has been lost, is > there a way to recover the bucket? > > I searched through the archives but didn't find anything exactly like this. > Please point me to the documentation if this has been seen before. > Do you have access to other buckets? Not sure what could have gone wrong, but start by looking at: $ radosgw-admin metadata get bucket:<bucket> and then, using the bucket_id from this command: $ radosgw-admin metadata get bucket.instance:<bucket>:<bucket-id> Yehuda _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |