On 04/25/11 12:13, Alex Williamson wrote: >> So, basically, 192.168.102 is the network where the VMs have a VF, and >> 192.168.103 is the network where the VMs use virtio for networking. >> >> The netperf commands are all run on either Host-A or VM-C: >> >> netperf -H $ip -jcC -v 2 -t TCP_RR -- -r 1024 -D L,R >> netperf -H $ip -jcC -v 2 -t TCP_STREAM -- -m 1024 -D L,R >> >> >> latency throughput >> (usec) Mbps >> cross-host: >> A-B, eth2 185 932 >> A-B, eth3 185 935 > > This is actually PF-PF, right? It would be interesting to load igbvf on > the hosts and determine VF-VF latency as well. yes, PF-PF. eth3 has the added bridge layer, but from what I can see the overhead is noise. I added host-to-host to put the host-to-VM numbers in perspective. > >> same host, host-VM: >> A-C, using VF 488 1085 (seen as high as 1280's) >> A-C, virtio 150 4282 > > We know virtio has a shorter path for this test. No complaints about the throughput numbers; the latency is the problem. > >> cross-host, host-VM: >> A-D, VF 489 938 >> A-D, virtio 288 889 >> >> cross-host, VM-VM: >> C-D, VF 488 934 >> C-D, virtio 490 933 >> >> >> While throughput for VFs is fine (near line-rate when crossing hosts), > > FWIW, it's not too difficult to get line rate on a 1Gbps network, even > some of the emulated NICs can do it. There will be a difference in host > CPU power to get it though, where it should theoretically be emulated > > virtio > pci-assign. 10GB is the goal; 1GB offers a cheaper learning environment. ;-) > >> the latency is horrible. Any options to improve that? > > If you don't mind testing, I'd like to see VF-VF between the hosts (to > do this, don't assign eth2 an IP, just make sure it's up, then load the > igbvf driver on the host and assign an IP to one of the VFs associated > with the eth2 PF), and cross host testing using the PF for the guest > instead of the VF. This should help narrow down how much of the latency > is due to using the VF vs the PF, since all of the virtio tests are > using the PF. I've been suspicious that the VF adds some latency, but > haven't had a good test setup (or time) to dig very deep into it. It's a quad nic, so I left eth2 and eth3 alone and added the VF-VF test using VFs on eth4. Indeed latency is 488 usec and throughput is 925 Mbps. This is host-to-host using VFs. David > Thanks, > > Alex > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
