On 06/11/2012 11:34 AM, Whit Blauvelt wrote: > On Sat, Jun 02, 2012 at 11:43:34PM -0400, Andrew Cathrow wrote: > >>> On Jun 1, 2012 2:17 PM, "Whit Blauvelt" < whit.virt@xxxxxxxxxxxxx > >>> wrote: >>>> I recall the kernel recently adding a new alternative for handling VM >>>> interfaces, but can't recall what it was called. Would that be a >>>> marked improvement? Is it supported by libvirt? >>> Linux Bridge < macvtap < SR-IOV >> >> or openvswitch > Looking into background info on these, it looks like SR-IOV capability is > specific to certain NICs and not documented for KVM/libvirt in any obvious > place. Maybe I just didn't find it. In a later message you indicate that your host has an Intel 82576 NIC. That *is* one of the few that has SR-IOV capabilities - each of its ports has up to 7 Virtual Functions (VF) which can each be directly assigned to a guest. The problem with this in the version of libvirt you use (0.8.3, which is very dated) is that you can only attach the device using <hostdev>, which doesn't support setting the MAC address of the device before assigning it to the guest; since SR-IOV VFs are each given new random MAC addresses on each reboot of the host, this means your guest will not be see a stable MAC address, and will thus believe that a new network device has been added after each host reboot. You can solve this with a short script to set the VFs' MAC addresses at host boot time, or if you can upgrade to libvirt-0.9.11 or later, you can use <interface type='hostdev'> to attach SR-IOV VFs to your guests: http://wiki.libvirt.org/page/Networking#PCI_Passthrough_of_host_network_devices (it also points to a page that explains how to use <hostdev> to attach an SR-IOV (or non-SR-IOV) NIC to a guest. > Openvswitch looks more widely useable and promising, but also lacking > user-level documentation. Openvswitch.org's "Documentation" section has > three brief notes, all presuming you've already got it in use. openvswitch itself is a fairly new project, and its support in libvirt is even newer. At this point, the libvirt support requires that an openvswitch bridge is already configured and running on the host. (I'm sure openvswitch would be happy to receive updates/additions to their documentation :-) Of course, since you're using libvirt-0.8.3, you wouldn't be able to easily use openvswitch with your guests anyway (and since you are using kernel-2.6.35 you wouldn't have openvswitch support in the kernel anyway, unless you add it yourself). > > Okay, so falling back to macvtap, a long post from over a year ago at > > http://ubuntuforums.org/showthread.php?t=1687750 > > says that the host cannot communicate directly with the guests through the > bridge when using it. Is this correct? That would rule it out for my use. Yes, that is true. There is a way around it, though - add a 2nd interface to each guest that is connected to an isolated bridge: http://wiki.libvirt.org/page/Guest_can_reach_outside_network%2C_but_can%27t_reach_host_%28macvtap%29 I'm not really sure how much performance you would gain from switch to macvtap though (I have heard of someone who improved lxc guest network performance from 4Gb/sec to 9Gb/sec by using macvtap to connect to a vlan interface (which I previously didn't even know was supported!), but that situation is very different from yours, and frankly the low numbers you give make it sound like you have some other basic problem that won't be fixed by switching to a different connection type. (Another thing to think about with macvtap is that it was only added in kernel-2.6.34, so the version in your kernel probably still has bugs.) > So now I fall back to the question of whether I should enable virtio where > I've currently got (working, but slowish) bridges defined. Yes. Any time the guest supports virtio-net, use it (unless you are using PCI passthrough to directly use the physical device). Note that the use of the guest virtio driver is completely orthogonal to whether you use macvtap, a host bridge, or a libvirt-managed virtual network > (I've got virtio > in use for memballoon, but not for the bridges.) I look at > http://wiki.libvirt.org/page/Virtio and while helpful it underspecifies: > > In the <interface> section, add a virtio model, like this: > > <interface type='network'> > ... > <model type='virtio' /> > </interface> > > That's all it says. So ... I've got entries that currently look about like: > > <interface type='bridge'> > <mac address='52:54:00:25:0a:a2'/> > <source bridge='br0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> > </interface> > > What if anything should carry across to the elipsis in the doc? That <interface> definition will use the default model for the hypervisor, which in this case is rtl8139. To change an existing interface definition to use a different model, just add the <model ... > line and leave the rest of the <interface> section as it is. If you are adding a new interface, you don't need to add a <mac address=.../> or <adress type='pci' .../> line, as those are automatically generated and added for you (in particular, don't add the <address> line, because each guest device needs to have a unique address, and there are some odd rules about placement of some devices that libvirt knows about and accounts for). > The doc at > http://www.linux-kvm.org/page/Tuning_KVM is similarly terse, and puts it in > terms of the qemu command line, rather than libvirt XML, so not completely > helpful short of studying how libvirt translates the XML to that line. Ah, > here's a more useful doc at > http://wiki.libvirt.org/page/Networking#Guest_configuration_2: > > Guest configuration > > In order to let your virtual machines use this bridge, their configuration > should include the interface definition as described in Bridge to LAN. In > essence you are specifying the bridge name to connect to. Assuming a shared > physical device where the bridge is called "br0", the following guest XML > would be used: > > <interface type='bridge'> > <source bridge='br0'/> > <mac address='00:16:3e:1a:b3:4a'/> > <model type='virtio'/> # try this if you experience problems with VLANs > </interface> > > So the <model type='virtio'/> can just go under <interface type='bridge'>, > that doesn't need to change to <interface type='network'>. Right. Those two things are orthogonal. <model type='xxx'/> is defining what kind of device to present to the guest. The type='nnn' attribute directly in <interface> is defining how to connect the guest's network data stream to the physical network. > Then should the > current line like: > > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> > > stay or go? If this line is already there, don't remove it. If there isn't one, don't add it - an address appropriate for the situation will be automatically added. > > Do others share my perception that it's a shame there's so little interest > in writing good docs in this whole area? As much documentation as exists, there could still be a lot more, and it could stand to be better organized. We welcome any contributions to the libvirt wiki. Some specific resources that may be helpful to you: http://www.libvirt.org/formatdomain.html#elementsNICS (This is (or should be) a full reference of every option supported in <interface> definitions. Pay close attention to all of the "since x.x.x" tags - many features have been added to libvirt since 0.8.3, so a lot of what's there won't be supported by your host's libvirt anyway.) http://wiki.libvirt.org/page/Troubleshooting (in particular, there are several items in the middle of the list specifically dealing with networking problems) http://wiki.libvirt.org/page/Networking (I think you've already found that one). _______________________________________________ libvirt-users mailing list libvirt-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvirt-users
