Kwan Lowe wrote: > On Wed, Oct 6, 2010 at 12:23 PM, Patrick Bartek<bartek047@xxxxxxxxx> wrote: >> I would use dd to clone (or back up) an entire hard drive. Easier. You can even pipe it through gzip to get a compressed image file. > > I do exactly that... dd piped through gzip then push through an SSH > session to a remote where it's extracted and written on the fly to > another LV. I use it to move kvm LVs from machine to machine. Works > great, but there's not much error checking. If it's critical I do an > md5sum on the LVs at the end then compare. Unless you have a really fast CPU or a really slow network the compression it taking longer than the network transfer. And in many cases you don't need the encryption of ssh, it just uses more CPU. What to do? One thing which can help is to use the built-in compression of ssh, depending on your setup that might be faster than gzip. But having the compress and encrypt in two cores helps with CPU usage. Another, if you are going through a trusted physical network (like a piece of cable or internal network) is to use netcat (aka "nc") for a full speed unencrypted transfer. Use common sense, sometimes security is needed. Finally, the lzop program uses way less CPU per byte than gzip, its compression sucks, but if the compression is running slower than the network lzop is usually a win, and if you need encryption the compress and encrypt are in separate threads (and hopefully cores). Beware pipes, which are slow in general. Piping your dd into ssh can slow the transfer by a factor of five.[1] Finally, if you have a good connection, enable jumbo packets in the routing at both ends, that will buy you some improved network speed and lower overhead. I move a few TB between systems each week, once a week, so I've played with this a bit. [1] simple pipe test: dd if=/dev/zero of=/dev/null bs=1k count=2M cat /dev/zero | dd of=/dev/null bs=1k count=2M look at the transfer rates reported and the system time. -- Bill Davidsen <davidsen@xxxxxxx> "We have more to fear from the bungling of the incompetent than from the machinations of the wicked." - from Slashdot -- users mailing list users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
