On Fri, 7 Sep 2018 11:10:08 +1000 Cameron Simpson <cs@xxxxxxxxxx> wrote: > On 06Sep2018 19:20, Ranjan Maitra <maitra@xxxxxxxxx> wrote: > >I have two drives mounted on a F28 system. Both are identical 4TB drives. The > >second one is empty. I am concerned about the first one failing so would like > >to copy the contents (which are around 3.7 TB) to the second. > > > >What is the fastest way to copy the contents of the first drive to the second? > >I was using rsync, but is there a better way? > > I like "cp -a", it is faster than rsync. Rsync's strength is incremental > update: make a sweep afterwards with rsync to convince yourself it is correct. > > Even faster is a tar pipeline: > > cd /drive1 > tar cf - . | ( cd /drive2; tar xf - ) > > because both cp and rsync do one file at a time. There will inherently be small > pauses at each file boundary. Actually, rsync might stream a little. > > Using piped tars and many files, particularly many small files, the first tar > can get ahead of the second tar for better throughput - the data queued in the > pipe (which has a buffer, and a generous one on Linux) allows the first tar to > proceed until the pipe is full if the second tar is blocked. (The second tar > will of course be blocked writing to drive2, but it won't be blocked reading > from drive1 because the first tar can read followon files from drive1 which the > second tar reads from the pipe). > > However, if you're already a significant way through your copy you may as well > stick with rsync unless you can easily do things in chunks, as changing systems > means wasting time copying already copied data. Do a "df" and make an estimate. > > If there are still hours to go you could consider switching methods and doing > the uncopied subdirectories: > > cd /drive1 > tar cf - uncopieddir1 uncopieddir2 ... | ( cd /drive2 ; tar xf - ) > > and then come back with rsync afterwards to clean up the rest: > > rsync -iaP /drive1/ /drive2/ Wow! What a fabulous approach: it has already done at least 10 times more in 10 minutes using your tar set up than using rsync in over an hour. Very cool! Thanks again! Best wishes, Ranjan _______________________________________________ users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
