> Isn't this already done in a better way by 'dd' using 'zero' copy > mechanism ?? (assuming it is already using either 'mmap()' -> > 'write()' or 'splice()' as your proposed piped copy is doing a lot > of pointless memory copies and cache trashing. No, with directio there isn't "a lot of pointless memory copies and cache trashing". There are in fact fewer mempoy copies than there would be with a simple: dd if=thing of=thing See: http://www.ukuug.org/events/linux2001/papers/html/AArcangeli-o_direct.html The proof of the pudding is in the eating, and in fact that recipe is extremely fast, for this application (sequential copy of GBs from one device to another). I spent a couple of hours testing before spending hundreds of hours copying. My bottom line was speed, so I didn't even look at memory usage or CPU. I would't mind using an extra 64 MB in order to get my copying done 30 hours faster. -- Ray Morris support@bettercgi.com Strongbox - The next generation in site security: http://www.bettercgi.com/strongbox/ Throttlebox - Intelligent Bandwidth Control http://www.bettercgi.com/throttlebox/ Strongbox / Throttlebox affiliate program: http://www.bettercgi.com/affiliates/user/register.php On Thu, 31 Mar 2011 14:15:04 +0200 Zdenek Kabelac <zkabelac@redhat.com> wrote: > Dne 31.3.2011 02:27, Stuart D. Gathman napsal(a): > > On Wed, 30 Mar 2011, Ray Morris wrote: > > > >>> 1..how to pipe two copy together > >> > >> This is what I use after some experimenting. I've found > >> it's often far faster than the more obvious use of dd: > >> > >> nice -5 dd if=/dev/clones/from bs=64M iflag=direct | > >> dd of=/dev/scratch/to bs=64M oflag=direct > >> > >> I use 64 MB extents. For smaller extents, a matching dd > >> block size might be good. > > > > Thanks for that recipe. It ought to be in lvm2 as an lvcopy utility > > (dynamically determining extent size, etc). > > > > > Isn't this already done in a better way by 'dd' using 'zero' copy > mechanism ?? (assuming it is already using either 'mmap()' -> > 'write()' or 'splice()' as your proposed piped copy is doing a lot > of pointless memory copies and cache trashing. > > If it's still faster - then your kernel setting for buffering are > most probaly not tuned for the best performance. > > Zdenek > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
