On Fri, Nov 26, 2010 at 1:00 PM, Michael Tokarev <mjt@xxxxxxxxxx> wrote: > 26.11.2010 22:19, Freddie Cash wrote: >> Are there still known memory leaks in the virtio block and/or net >> drivers? ÂIf not, have we found one? >> >> This is our host setup: >> Â - Tyan h2000M motherboard >> Â - 2x dual-core AMD Opteron 2220 CPUs >> Â - 16 GB RAM >> Â - quad-port NIC bonded together into 1 kvmbr0 interface >> Â - 3Ware 9550SXU-12ML RAID controller >> Â - 12x 400 GB SATA drives (4-disk RAID5, 4-disk RAID5, 3-disk RAID5, >> spare disk) >> Â - LVM used to stitch the 3 arrays into one volume group >> >> Running 64-bit Debian 5.0 (Lenny), using qemu-kvm0.12.4 and kernel >> 2.6.32 from lenny-backports. > > There were some memleaks in i/o paths fixed in 0.12 series of kvm. > In particular, 0.12.4 upstream version had a memory leak in virtio_blk, > but that bug isn't present in 0.12.4+dfsg-1 - first 0.12.4 version as > appeared in debian (the fix included upstream in 0.12.5). ÂI know no > other memleaks in later 0.12 series which aren't fixed in debian. Okay, that's what I've read via the bug reports and changelogs, but wanted to be sure. >> Guests are Ubuntu Server 8.04 LTS, Debian Lenny, Windows XP, and Windows 2003. > > For debian lenny guests using virtio, consider switching to > backports 2.6.32 kernel too - lenny's 2.6.26 has several > defects in virtio implementation. Can do. There's only one that isn't already running the 2.6.32-bpo kernel. >> All of the Linux VMs use virtio for both network and disk. >> All the Windows VMs use virtio for network and IDE emulation for disk. >> >> Within 2 weeks of booting, the host machine is using 2 GB of swap, and >> disk I/O wait is through the roof. ÂRestarting all of the VMs will >> free up RAM, but restarting the whole box is the only way to get >> performance back up. >> >> A guest configured to use 8 GB of RAM will have 9 GB virt and 7.5 GB >> res shown in top. ÂIn fact, every single VM shows virt above the limit >> set for the VM. ÂUsually by close to 25%. >> >> Going back through the mailing list archives and bug archives, there's >> been mention of several memory leaks in the virtio drivers back to >> KVM-72. ÂLast bug report I read shows them all being fixed in 0.12.4, >> which we're running. > > Yes, 0.12.4 had that bug, but it's fixed in 0.12.4+dfsg-1. > The current version in backports is based on 0.12.5. That would be this version (which we have installed)? Version: 0.12.4+dfsg-1~bpo50+2 Hrm, no, there's a newer one, just not in our mirror yet: 0.12.5+dfsg-3~bpo50+2 I'll see if we can upgrade to that one over the weekend as well. > Well, how about reading the changelog first, before asking? They're > there for a reason, right? I read everything under /usr/share/doc/[qemu|kvm|libvirt], along with the release notes on the qemu/kvm websites, and ended up with more questions than answers, hence why I posted. :) I also read through a bunch of old threads on the kvm mailing list, several bug reports for qemu, qemu-kvm, libvirt, rhel, and so forth. The repeating theme through them all was that mem leaks in virtio cropped up every other release or so, including some major ones in the 0.12.x series (0.12.3 and 0.12.4 especially). The symptoms we're having now are the same ones we originally had with KVM-72 when we first switched from SCSI-emulated disks to virtio. > Speaking of your setup, in addition to what Stephen said already, > I'd strongly suggest considering hugepages - for this amount of > memory hugepages should help quite alot, and this way you'll eliminate > lots of pagetable entries. Hrm, I'll have to do some more googling on this. There's nothing about it in the man pages or --help output, but I've seen it references on the mailing list. Same for KSM (as suggested in another message), although would that be helpful with such a variety of VM guest OSes (Debian Etch, Debian Lenny, Ubuntu Server, Windows XP, Windows 2003)? -- Freddie Cash fjwcash@xxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
