hi folks, i'm new to linux lvm but a longtime user of lvm's on other commercial unices. i love some of the features like r/w snapshots! and indeed snapshots are the primary reason i'm even interested in using LVM on linux. however, snapshots really shoot my system in the foot. in my (12 processor, 12GB RAM, x86_64 centos 5.5) server, i have two pricey areca hardware raid cards that give me ridiculous write performance: i can sustain over 700MByte/sec write speeds (writing a file with dd if=/dev/zero bs=1M, twice the size of the raid card's onboard memory and timing the write + a sync afterwards) to the volumes on either card (both backed by 7 or more fast disks). enabling a single snapshot reduces that speed by a factor of 10! enabling more snapshots isn't as drastic but still doesn't really scale well: no snapshot = ~11sec (727MB/sec) 1 snapshot = ~102sec (78MB/sec) 2 snapshots = ~144sec (55MB/sec) 3 snapshots = ~313sec (25MB/sec) 4 snapshots = ~607sec (15MB/sec) i have my snapshots set up a separate array from the master filesystem, on a separate raid card. i did not change the default parameters for the setup (ie blocksize) because our typical workload is reading and writing small (<64k) files. i can copy non-snapshotted files from an LVM on array1 to array2 at a good clip, 307MByte/sec (including sync). copies from the parent array to itself (no snapshots enabled) go at about 220MByte/sec. all of my measurements were repeated 4x and averaged--occasionally there was one that was a good 30% faster than the other 3, but it was always an outlier. typically all measurements for a given scenario were within 10% of eachother. so i guess that as a replacement for a netapp, setup with a few hourly & daily, and even one weekly snapshot isn't something people do with stock linux LVM? or am i just doing it wrong? in searching the archives i heard about zumastor. is that really production-ready? the no-new-releases in the last 2 years and not being in the mainstream kernel makes me leery of it. i think we can live with the factor-of-10 performance degradation on a daily basis--we can turn off all the snapshots in case we really have to hammer the server (which has a 4Gbit uplink to a render farm, so it is possible for us to actually write over 70MByte/sec when things are humming, via NFS), and in general it serves SMB at closer to 400Mbit than 4000, so all the desktop users will not notice a difference. it seems that others have seen these problems: http://www.nikhef.nl/~dennisvd/lvmcrap.html as an example. any thoughts? thanks in advance for your input! _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
