Hi, I had been noticing similar behaviour recently and came to a similar conclusion. So I took a look into it and in my case it was actually came down to the fact that TSO/GSO is now implemented (and enabled) in most NIC drivers that support it. TSO/GSO has the effect of increasing the 'internal mtu'. tbf has a check in its enqueue function that checks see if the incoming packet is over the tc's mtu (which defaults to 2047) - but with TSO/GSO the 'packet size' grows larger than that. If you are seeing this issue then the simple fix was just to add the appropriate mtu parameter to the tc command - so in your case it would be (yu can try playing with the value of the mtu): tc qdisc add dev eth4 root handle 1: tbf rate 6300kbit burst 100kb latency 200ms mpu 64 mtu 10000 To verify whether it is the TSO/GSO causing the issue you could try disabling them (though you will lose some of the performance benefits) using ethtool -K eth4 tso off gso off - if you do this then your original tbf configuration should work ok. Incidentally this effect is much worse when tbf is used on the loopback interface which has a default interface mtu of 16K - thus you need to set the tc mtu to about 100k. For a bit more on it see my post: http://timesinker.blogspot.com/2011/02/using-linux-tbf-qdisc-for-rate-limiting.html Cheers, Piers O'Hanlon On 12 March 2011 16:59, Tom Eastep <teastep@xxxxxxxxxxxxx> wrote: > It appears that the breakage occurred somewhere between 2.6.26 and > 2.6.32. I've tried this on recent X86 Fedora, OpenSuSE and Debian > releases and the results are similar. In all cases, X86 works as expected. > > Here's a test on 2.6.26: > > sami:~# uname -r > 2.6.26-2-vserver-amd64 > sami:~# scp shorewall-docs-xml-4.4.17.tgz gateway: > shorewall-docs-xml-4.4.17.tgz 100% 6306KB 6.2MB/s 00:00 > > sami:~# tc qdisc add dev eth4 root handle 1: tbf rate 6300kbit \ > burst 100kb latency 200ms mpu 64 > > sami:~# scp shorewall-docs-xml-4.4.17.tgz gateway: > shorewall-docs-xml-4.4.17.tgz 100% 6306KB 1.0MB/s 00:06 > > sami:~# tc -s qdisc ls dev eth4; tc -s class ls dev eth4 > qdisc tbf 1: root rate 6300Kbit burst 100Kb lat 200.0ms > Sent 6919034 bytes 6366 pkt (dropped 4, overlimits 8825 requeues 0) > rate 0bit 0pps backlog 0b 0p requeues 0 > class tbf 1:1 parent 1: > sami:~# > > This test is from 2.6.32: > > #gateway:~# uname -r > 2.6.32-5-vserver-amd64 > > root@gateway:~# scp shorewall-docs-xml-4.4.17.tgz sami: > shorewall-docs-xml-4.4.17.tgz 100% 6306KB 6.2MB/s 00:00 > > root@gateway:~# scp shorewall-docs-xml-4.4.17.tgz sami: > shorewall-docs-xml-4.4.17.tgz 100% 6306KB 6.2MB/s 00:00 > root@gateway:~# tc qdisc add dev eth4 root handle 1: tbf rate \ > 6300kbit burst 100kb latency 200ms mpu 64 > root@gateway:~# scp shorewall-docs-xml-4.4.17.tgz sami: > shorewall-docs-xml-4.4.17.tgz 38% 2448KB 30.8KB/s - stalled - > shorewall-docs-xml-4.4.17.tgz 38% 2448KB 27.7KB/s - stalled - > > At this point I killed scp. > > root@gateway:~# tc -s qdisc ls dev eth1; tc -s class ls dev eth1 > qdisc tbf 1: root refcnt 6 rate 6300Kbit burst 100Kb lat 200.0ms > Sent 1097211 bytes 1347 pkt (dropped 285, overlimits 0 requeues 0) > rate 0bit 0pps backlog 0b 0p requeues 0 > class tbf 1:1 parent 1: > > root@gateway:~# > > -Tom > -- > Tom Eastep \ When I die, I want to go like my Grandfather who > Shoreline, \ died peacefully in his sleep. Not screaming like > Washington, USA \ all of the passengers in his car > http://shorewall.net \________________________________________________ > > -- To unsubscribe from this list: send the line "unsubscribe linux-net" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
