At Clyso we've been building a tool that can migrate S3 data around
called Chorus. Normally I wouldn't promote it here, but it's open
source and sounds like it might be useful in this case. I don't work on
it myself, but thought I'd mention it:
https://github.com/clyso/chorus
One problem with rclone is that it doesn't support copying ACLs AFAIK.
There was an incomplete PR for that a while back that never got merged.
We did some work on that as well, though I'm not sure if it's been up
streamed yet.
Mark
On 4/11/24 16:29, Casey Bodley wrote:
unfortunately, this cloud sync module only exports data from ceph to a
remote s3 endpoint, not the other way around:
"This module syncs zone data to a remote cloud service. The sync is
unidirectional; data is not synced back from the remote zone."
i believe that rclone supports copying from one s3 endpoint to
another. does anyone have experience with that?
On Thu, Apr 11, 2024 at 4:45 PM James McClune <mcclune.789@xxxxxxxxx> wrote:
Hello Ceph User Community,
I currently have a large Amazon S3 environment with terabytes of data
spread over dozens of buckets. I'm looking to migrate from Amazon S3 to an
on-site Ceph cluster using the RGW. I'm trying to figure out the
most efficient way to achieve this. Looking through the documentation, I
found articles related to the cloud sync module, released in Mimic (
https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/). I also watched
a video on the cloud sync module as well. It *sounds* like this is the
functionality I'm looking for.
Given I'm moving away from Amazon S3, I'm really just looking for a one-way
replication between the buckets (i.e. Provide an Amazon S3 access
key/secret which is associated to the buckets and the same for the Ceph
environment, so object data can be replicated one-to-one, without creating
ad-hoc tooling). Once the data is replicated from S3 to Ceph, I plan on
modifying my boto connection objects to use the new Ceph environment. Is
what I'm describing feasible with the cloud sync module? Just looking for
some affirmation, given I'm not well versed in Ceph's RGW, especially
around multi-site configurations.
Thanks,
Jimmy
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
