A PR that adds experimental support for restoring a bucket index was merged into main. It’ll need back ports to reef, quincy, and pacific. https://tracker.ceph.com/issues/59053 <https://tracker.ceph.com/issues/59053?issue_count=48&issue_position=1&next_issue_id=58780> Currently it does not work for versioned buckets. And it is experimental. If anyone is able to try it I’d be curious about your experiences. Eric (he/him) > On Feb 23, 2023, at 11:20 AM, J. Eric Ivancich <ivancich@xxxxxxxxxx> wrote: > > Off the top of my head: > > 1. The command would take a bucket marker and a bucket names as arguments. It might also need some additional metadata to fill in gaps. > 2. Scan the data pool for head objects that refer to that bucket marker. > 3. Based on the number of such objects found, create a bucket index with an appropriate number of shards (approximately that number divided by 50,000). > 4. For each such object: > a. Read the manifest in the head object and see if all tail objects exit. > b. If any tail objects are missing, maybe report that object on the console as non-recoverable? > c. If all tail objects present, resolve the name of the object from the head object and add the bucket index entry to the appropriate shard. > > Note 1: Slight variations may be needed depending on whether or not the bucket entry object exists and the bucket instance object exists. > Note 2: Versioned buckets will likely require some additional steps, but I’d need to refresh my memory on some of the details. > > Eric > (he/him) > >> On Feb 23, 2023, at 4:51 AM, Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx> wrote: >> >> Hi, >> >> On 22.02.23 17:45, J. Eric Ivancich wrote: >> >>> You also asked why there’s not a command to scan the data pool and recreate the bucket index. I think the concept would work as all head objects include the bucket marker in their names. There might be some corner cases where it’d partially fail, such as (possibly) transactional changes that were underway when the bucket index was purged. And there is metadata in the bucket index that’s not stored in the objects, so it would have to be recreated somehow. But no one has written it yet. >> >> I am not in an urgent need to get such a feature. >> >> How would the process look to get development started in this direction? >> >> Regards >> -- >> Robert Sander >> Heinlein Consulting GmbH >> Schwedter Str. 8/9b, 10119 Berlin >> >> https://www.heinlein-support.de >> >> Tel: 030 / 405051-43 >> Fax: 030 / 405051-19 >> >> Amtsgericht Berlin-Charlottenburg - HRB 220009 B >> Geschäftsführer: Peer Heinlein - Sitz: Berlin >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |