> If you are experiencing significant performance degradation with > usage of single 'dd' with two different driver compared with 2 > piped 'dd' - you should probably make a report on lkml list - There > is something broken if you gain 'hours' like you said by piping 2 dd > commands. I'm taking this off list as off topic, so if anyone else is interested please let me know and I'll update you on what we find. -- Ray Morris support@bettercgi.com Strongbox - The next generation in site security: http://www.bettercgi.com/strongbox/ Throttlebox - Intelligent Bandwidth Control http://www.bettercgi.com/throttlebox/ Strongbox / Throttlebox affiliate program: http://www.bettercgi.com/affiliates/user/register.php On Tue, 05 Apr 2011 23:10:20 +0200 Zdenek Kabelac <zkabelac@redhat.com> wrote: > Dne 5.4.2011 18:25, Ray Morris napsal(a): > >> Isn't this already done in a better way by 'dd' using 'zero' copy > >> mechanism ?? (assuming it is already using either 'mmap()' -> > >> 'write()' or 'splice()' as your proposed piped copy is doing a lot > >> of pointless memory copies and cache trashing. > > > > No, with directio there isn't "a lot of pointless memory copies > > and cache trashing". There are in fact fewer mempoy copies than > > there would be with a simple: > > dd if=thing of=thing > > > > See: > > http://www.ukuug.org/events/linux2001/papers/html/AArcangeli-o_direct.html > > > > Seem like 10 years old paper for 2.4.6 kernel - so I do not think it > still applies to upstream 2.6.38 - thought I've not made any recent > measuring myself. > > > The proof of the pudding is in the eating, and in fact that > > recipe is extremely fast, for this application (sequential > > copy of GBs from one device to another). I spent a couple > > of hours testing before spending hundreds of hours copying. > > My bottom line was speed, so I didn't even look at memory > > usage or CPU. I would't mind using an extra 64 MB in order > > to get my copying done 30 hours faster. > > The kernel should be able to use as much free memory as you have - and > when it's properly configured - disk flushing running in separate > thread should ensure maximal throughput. > > If you are experiencing significant performance degradation with > usage of single 'dd' with two different driver compared with 2 > piped 'dd' - you should probably make a report on lkml list - There > is something broken if you gain 'hours' like you said by piping 2 dd > commands. > > (I don't have fast enough arrays myself for such playing) > > Zdenek > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
