Hello, I have two KVM VMs running on two servers, which are connected to the same physical switch. Each server has a 10G port connected to the switch, which have two VLANs. For the first vlan, I directly configured IPs on the vlan interfaces on the two servers. With this vlan, the two servers can use iperf to push 600Mbps UDP packets to each other with little loss: [ 3] 0.0- 2.0 sec 147 MBytes 617 Mbits/sec 0.017 ms 257/105263 (0.24%) [ 3] 2.0- 4.0 sec 147 MBytes 618 Mbits/sec 0.019 ms 0/105133 (0%) [ 3] 4.0- 6.0 sec 148 MBytes 619 Mbits/sec 0.019 ms 0/105244 (0%) [ 3] 6.0- 8.0 sec 148 MBytes 619 Mbits/sec 0.022 ms 0/105237 (0%) [ 3] 8.0-10.0 sec 148 MBytes 619 Mbits/sec 0.018 ms 0/105254 (0%) ... [ 3] 0.0-30.0 sec 2.16 GBytes 619 Mbits/sec 0.018 ms 256/1578752 (0.016%) (Not sure what happend in the first 2 seconds...) For the two VMs, one is using 100M VNIC, the other is using 1000M one. The vnet interfaces for the two VMs are put on two bridges on the two servers, both tap into the second vlan. I then run iperf to send UDP packets from the 100M VM to the 1000M VM using the following parameter: iperf -c 10.6.6.17 -t 30 -i 2 -r -b 600M And I see something like this: [ 4] 0.0- 1.0 sec 42.6 MBytes 358 Mbits/sec 0.046 ms 93/30508 (0.3%) [ 4] 1.0- 2.0 sec 42.9 MBytes 360 Mbits/sec 0.044 ms 0/30613 (0%) [ 4] 2.0- 3.0 sec 41.9 MBytes 352 Mbits/sec 0.041 ms 507/30420 (1.7%) [ 4] 3.0- 4.0 sec 41.9 MBytes 352 Mbits/sec 0.039 ms 687/30591 (2.2%) [ 4] 4.0- 5.0 sec 48.7 MBytes 408 Mbits/sec 0.037 ms 0/34704 (0%) [ 4] 5.0- 6.0 sec 50.5 MBytes 423 Mbits/sec 0.041 ms 0/36010 (0%) [ 4] 6.0- 7.0 sec 21.6 MBytes 181 Mbits/sec 0.064 ms 4023/19431 (21%) [ 4] 7.0- 8.0 sec 24.5 MBytes 206 Mbits/sec 0.225 ms 1501/19003 (7.9%) [ 4] 8.0- 9.0 sec 24.2 MBytes 203 Mbits/sec 0.157 ms 2541/19786 (13%) [ 4] 9.0-10.0 sec 24.1 MBytes 202 Mbits/sec 0.102 ms 2670/19859 (13%) [ 4] 10.0-11.0 sec 21.6 MBytes 181 Mbits/sec 0.273 ms 3672/19078 (19%) [ 4] 11.0-12.0 sec 20.7 MBytes 174 Mbits/sec 0.061 ms 5038/19811 (25%) [ 4] 12.0-13.0 sec 24.3 MBytes 204 Mbits/sec 0.137 ms 2096/19403 (11%) [ 4] 13.0-14.0 sec 21.2 MBytes 178 Mbits/sec 0.381 ms 4123/19252 (21%) [ 4] 14.0-15.0 sec 46.0 MBytes 386 Mbits/sec 0.047 ms 347/33174 (1%) [ 4] 15.0-16.0 sec 47.6 MBytes 399 Mbits/sec 0.039 ms 0/33936 (0%) [ 4] 16.0-17.0 sec 47.3 MBytes 397 Mbits/sec 0.040 ms 0/33750 (0%) [ 4] 17.0-18.0 sec 46.0 MBytes 386 Mbits/sec 0.049 ms 191/33005 (0.58%) [ 4] 18.0-19.0 sec 46.2 MBytes 387 Mbits/sec 0.051 ms 0/32923 (0%) [ 4] 19.0-20.0 sec 46.2 MBytes 387 Mbits/sec 0.049 ms 0/32942 (0%) [ 4] 20.0-21.0 sec 45.9 MBytes 385 Mbits/sec 0.032 ms 0/32711 (0%) [ 4] 21.0-22.0 sec 45.9 MBytes 385 Mbits/sec 0.037 ms 37/32769 (0.11%) [ 4] 22.0-23.0 sec 45.0 MBytes 378 Mbits/sec 0.052 ms 83/32205 (0.26%) [ 4] 23.0-24.0 sec 45.1 MBytes 378 Mbits/sec 0.039 ms 0/32180 (0%) [ 4] 24.0-25.0 sec 34.6 MBytes 290 Mbits/sec 0.051 ms 0/24689 (0%) [ 4] 25.0-26.0 sec 33.8 MBytes 284 Mbits/sec 0.047 ms 0/24132 (0%) [ 4] 26.0-27.0 sec 33.9 MBytes 285 Mbits/sec 0.050 ms 0/24208 (0%) [ 4] 27.0-28.0 sec 33.6 MBytes 282 Mbits/sec 0.057 ms 0/23994 (0%) [ 4] 28.0-29.0 sec 40.1 MBytes 337 Mbits/sec 0.040 ms 0/28616 (0%) [ 4] 0.0-30.0 sec 1.11 GBytes 318 Mbits/sec 0.038 ms 27608/838083 (3.3%) So, my question is... Why do I see this kind of variable dropping rate? Is it because of KVM doing some kind of rate throttling to emulate the virtual nic speed? Is there a way to achieve a more consistent dropping rate? The biggest problem I am having is actually with TCP. For the first several seconds, the drop rate is small enough for TCP to ramp up to a good speed, but then the drop rate becomes larger and essentially stalls my TCP. Here is what I am talking about: [ 4] 12.0-14.0 sec 26.0 MBytes 109 Mbits/sec [ 4] 14.0-16.0 sec 18.0 MBytes 75.5 Mbits/sec --> start to drop [ 4] 16.0-18.0 sec 0.00 Bytes 0.00 bits/sec --> stalled [ 4] 18.0-20.0 sec 47.9 MBytes 201 Mbits/sec --> back to speed [ 4] 20.0-22.0 sec 58.5 MBytes 245 Mbits/sec [ 4] 22.0-24.0 sec 44.6 MBytes 187 Mbits/sec Thanks. -Simon -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
