I have done 700k and 800k files transfers (including hardlinks), but indeed it could take a while to compute the transferlist. Newer rsync versions bring down the amount of memory needed drastically. That is one of the reasons I offer a recent rsync in RPMforge. There is almost never a good reason to use a dated rsync.
I just thought I'd de-lurk and chime in that there are some patches for ssh to allow better performance:
http://www.psc.edu/networking/projects/hpn-ssh/
If you do end up using rsync for something like this via ssh, you might want to look at some of the Pittsburgh Supercomputing Center's patches. The high-performance patches can allow you to see dramatic increases in throughput.
_______________________________________________ CentOS mailing list CentOS@xxxxxxxxxx http://lists.centos.org/mailman/listinfo/centos
