On Wed, 2011-04-20 at 19:57 -0600, David Ahern wrote:
> In general should virtual functions outperform virtio+vhost for
> networking performance - latency and throughput?
>
> I have 2 VMs running on a host. Each VM has 2 nics -- one tied to a VF
> and the other going through virtio and a tap device like so:
>
> ------ ----
> | |----------------| VF |---
> | | ---- |
> | VM 1 | |
> | | ----- |
> | |---| tap |--- |
> ------ ----- | ---
> --- | e |
> | b | | t |
> | r | | h |
> --- | 2 |
> ------ ----- | ---
> | |---| tap |--- |
> | | ----- |
> | VM 2 | |
> | | ---- |
> | |----------------| VF |---
> ------ ----
>
> The network arguments to qemu-kvm are:
> -netdev type=tap,vhost=on,ifname=tap2,id=netdev1
> -device virtio-net-pci,mac=${mac},netdev=netdev1
>
> where ${mac} is unique to each VM and for the VF:
> -device pci-assign,host=${pciid}
>
> netserver is running within the VMs, and the netperf commands I am
> running are:
>
> netperf -p 12346 -H <ip> -l 20 -jcC -fM -v 2 -t TCP_RR -- -r 1024
> netperf -p 12346 -H <ip> -l 20 -jcC -fM -v 2 -t TCP_STREAM
>
> where <ip> changes depending on which interface I want to send the
> traffic through. To say the least results are a bit disappointing for
> the VF:
>
> latency throughput
> (usec/Tran) (MB/sec)
> Host-VM
> over virtio 139.160 1199.40
> over VF 488.124 209.22
>
> VM-VM
> over virtio 322.056 773.54
> over VF 488.051 328.88
>
> I am just getting started with VFs and could use some hints on how to
> improve the performance.
Device assignment via a VF provides the lowest latency and most
bandwidth for *getting data off the host system*, though virtio/vhost is
getting better. If all you care about is VM-VM on the same host or
VM-host, then virtio is only limited by memory bandwidth/latency and
host processor cycles. Your processor has 25GB/s of memory bandwidth.
On the other hand, the VF has to send data all the way out to the wire
and all the way back up through the NIC to get to the other VM/host.
You're using a 1Gb/s NIC. Your results actually seem to indicate you're
getting better than wire rate, so maybe you're only passing through an
internal switch on the NIC, in any case, VFs are not optimal for
communication within the same physical system. They are optimal for off
host communication. Thanks,
Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
