We have been using Amazon S3 (rclone.org)<https://rclone.org/s3/#ceph> to copy all the data to a filesystem nightly to provide an S3 backup mechanism. It has Ceph support out the box (added by one of my colleagues a few years ago). ________________________________ From: Adam Prycki <aprycki@xxxxxxxxxxxxx> Sent: Wednesday, September 25, 2024 7:09 PM To: Shilpa Manjrabad Jagannath <smanjara@xxxxxxxxxx> Cc: ceph-users@xxxxxxx <ceph-users@xxxxxxx> Subject: [EXTERNAL] Re: Backup strategies for rgw s3 [You don't often get email from aprycki@xxxxxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Yes, I know. It's just that I would need to define zone wide default lifecycle. For example, archivezone stores 30 days of object versions unless specified otherwise. Is there a way to do it? As far as I know lifecycle you linked is configured per bucket. As a small cloud provide we cannot really configure lifecycle policies for users. Adam Prycki On 25.09.2024 19:10, Shilpa Manjrabad Jagannath wrote: > starting from quincy, you can define rules for lifecycle to execute on > Archive zone alone by specifying > <ArchiveZone/> flag under <Filter> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftracker.ceph.com%2Fissues%2F53361&data=05%7C02%7Calexhus%40microsoft.com%7C96c97b0ee50b44ab1f7d08dcdd8d52dc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638628846286567323%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=0IH5xTWHvQYqdqMtukhoTCwN4hVAQ5ineog2%2B9HKkec%3D&reserved=0<https://tracker.ceph.com/issues/53361> > > > On Wed, Sep 25, 2024 at 7:59 AM Adam Prycki <aprycki@xxxxxxxxxxxxx> wrote: > >> Hi, >> >> I'm currently working on a project which requires us to backup 2 >> separate s3 zones/realms and retain it for few months. Requirements were >> written by someone who doesn't know ceph rgw capabilities. >> We have to do incremental and full backups. Each type of backup has >> separate retention period. >> >> Is there a way to accomplish this with in a sensible way? >> >> My fist idea would be to create multisite replication to archive-zone. >> But I cannot really enforce data retention on archive zone. It would >> require us to overwrite lifecycle policies created by our users. >> As far as I know it's not possible to create zone level lifecycle >> policy. Users get their accounts are provisioned via openstack swift. >> >> Second idea would be to create custom backup script and copy all the >> buckets in the cluster to different s3 zone. Destination buckets could >> be all versioned to have desired retention. But this option feels very >> hackish and messy. Backing up 2 separate s3 zones to one could cause >> collision in bucket names. Prefixing bucket names with additional >> information is not safe because buckets have fixed name length. >> Prefixing object key name is also not ideal. >> >> Best regards >> Adam Prycki >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |