Re: Migrate Individual Buckets

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Den mån 12 dec. 2022 kl 21:18 skrev Benjamin.Zieglmeier
<Benjamin.Zieglmeier@xxxxxxxxxx>:
> We are in the process of building new stage (non-production) Ceph RGW clusters hosting s3 buckets. We are looking to have our customers migrate their non-production buckets to these new clusters. We want to help ease the migration, to hopefully improve adoption, and I wanted to ask the group if anyone had a slick way of forklifting a given s3 bucket from one RGW Ceph cluster to another. We plan to have the same versions (Luminous 12.2.11) across both clusters (to start).
>
> I don’t want to leverage multisite sync, as we want to be able to move buckets individually, and will not be moving all buckets. I thought about building a python script to do the move, as `aws s3 sync` would do most of the heavy lifting, but I want to bring along other artifacts, such as: Lifecycle configurations, versioning, bucket policies and/or ACLs, etc.

I haven't studied this last part on moving policies and so on, but I
would take a look at rclone, it has tons of options for s3 endpoints,
and does really well in the parallelism department to move large
amounts of data fast if you want to do it from the outside as opposed
to doing in-ceph syncs. Though I think you can do per-bucket sync with
ceph also.

-- 
May the most significant bit of your life be positive.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux