Re: Migrating from S3 to Ceph RGW (Cloud Sync Module)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Le jeudi 11 avril 2024, 23:44:05 CEST Gilles Mocellin a écrit :
> Le jeudi 11 avril 2024, 23:29:30 CEST Casey Bodley a écrit :
> 
> > unfortunately, this cloud sync module only exports data from ceph to a
> > remote s3 endpoint, not the other way around:
> > 
> > "This module syncs zone data to a remote cloud service. The sync is
> > unidirectional; data is not synced back from the remote zone."
> > 
> > i believe that rclone supports copying from one s3 endpoint to
> > another. does anyone have experience with that?
> 
> 
> Hello,
> 
> Yes, rclone can do that, and seems too have good performances (I didn't try
> 
 something else).
> You can (should) parallelize (--transfers), use many options, like rsync...
> 
 use checksums, launch several time (sync not just copy).
> 
> Of course, the machine where you launch the copy/sync, should be well 
> connected (good bandwidth) with either the source and the destination.
> 
> You can also launch several rclone, for specific buckets, from different 
> machines, in parallel...

Ah, and of course, if you use versioning, lifecycle rules, bucket policies, or 
all other "metadata" other than objects in buckets,
rclone will NOT copy them. You should re-create (or something like export / 
import) with awscli or s3cmd.

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux