Re: HTB and PRIO qdiscs introducing extra latency when output interface is saturated

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Andy, Many thanks for your reply. Below is some output from the queueing
disciplines to show that the filters are working correctly and they are
going to the right classes.

NOTE: The root qdisc of each interface is deleted before I run the
tests. This resets the statistics for the qdisc. The following is the
output after the tests.


Output of tc -s qdisc show on the core route for the 3 network
interfaces.


qdisc dsmark 1: dev eth0 indices 0x0040 set_tc_index
 Sent 2183574289 bytes 1496372 pkts (dropped 60982, overlimits 0
requeues 0)

qdisc htb 2: dev eth0 parent 1: r2q 10 default 0 direct_packets_stat 22
 Sent 2183574289 bytes 1496372 pkts (dropped 60982, overlimits 140759
requeues 0)

qdisc pfifo 8007: dev eth0 parent 2:10 limit 5p
 Sent 7265998 bytes 51169 pkts (dropped 0, overlimits 0 requeues 0)

qdisc red 8008: dev eth0 parent 2:20 limit 60Kb min 15Kb max 45Kb
 Sent 2176307367 bytes 1445181 pkts (dropped 60982, overlimits 60982
requeues 0)
  marked 0 early 60982 pdrop 0 other 0

qdisc pfifo 8009: dev eth1 limit 1000p
 Sent 33334496 bytes 477176 pkts (dropped 0, overlimits 0 requeues 0)

qdisc pfifo 800a: dev eth2 limit 1000p
 Sent 40637134 bytes 585931 pkts (dropped 0, overlimits 0 requeues 0)


Again here is the ASCII diagram

(network A)  -->  (eth1) core router (eth0) --> (network C)
                            (eth2) 
			     ^
                             |
                             |
                         (network B)



>From network A to C (from 2 pcs used for the purpose of traffic
generation)

TCP traffic - pc 1
Sent 994762580 bytes 658704 pkts (dropped 0, overlimits 0 requeues 0)

VoIP traffic - pc 2
Sent 7286487 bytes 51298 pkts (dropped 0, overlimits 0 requeues 0)

>From network B to C
TCP traffic
Sent 1271745729 bytes 841217 pkts (dropped 27, overlimits 0 requeues 0)


So total amount of packets transmitted to incoming interface on the core
router is 658704 + 51298 + (841217 - 27) = 1,551,192 packets.

The total sent by the dsmark and htb qdisc on the core router is
1,496,372 packets and 60,982 are dropped. The total received is
1,557,354. There is also some more traffic received from other nodes in
network A, but this is minimal and also traffic from the core router
itself. This should account for the difference.

VoIP traffic sent from a machine in network A = 51298 packets. It is
practically the same as the number of packets that pass through the
pfifo 51169 which is attached to class 2:1


TCP traffic that should be passing through class 2:10 which is the BE
class is 658,704 packets (TCP) from network A and 841,217 packets from B
which equals which totals 1,499,921

traffic sent from the BE class is  1,445,181 + 60,982 packets which were
dropped. So 1,506,163 packets were received by the BE class 2:10

The traffic sent from the output interface of eth1 and eth2 is mainly
acks back to network A and network B respectively. 


>100mbit will be too high if it's 100mbit nic.

What value would you recommend to set as the ceil for a 100 mbit NIC ??.


>Don't know what pass_on will mean here.

pass_on means if no class id equal to the result of the filter is found
then try next filter, which is the BE class in this case.



So back to the main question, could anyone tell me why the delay is so
high (30ms) for VoIP packets which are treated with the EF phb when the
outgoing interface of core  router to network c is saturated ?



Jonathan




> Won't hurt if the packets are small voip but TBF has a nasty habit of 
> taking 1 from the burst/mtu you specify so your burst setting may result 
> in packets >1499B getting dropped - tc -s -d qdisc ls dev ... should 
> show what it's using.

> Without wrapping it with something like htb red won't shape traffic.

I am not to concerned about the PRIO + TBF setup. My priority is with
the htb setup but I will look into this and see if I notice that.


On Wed, 2005-07-27 at 14:25 +0100, Andy Furniss wrote:
> Jonathan Lynch wrote:
> 
> > Could anyone tell me why the delay is so high (30ms) for VoIP packets
> > which are treated with the EF phb when the outgoing interface of core
> > router to network c is saturated ?
> > 
> 
> I have never used dsmark so am not sure about the classification parts 
> of your rules. You need to check where the packets are going with with 
> tc -s qdisc ls dev ...
> 
> The other parts have some issues see below.
> 
> > -----------------------------------------------------------------------------------------------
> > The config I used for each setup is included below. These are slight
> > modifications that are supplied with iproute2 source code.
> > 
> > Config 1 using htb
> > -------------------
> > tc qdisc add dev $1 handle 1:0 root dsmark indices 64 set_tc_index
> > tc filter add dev $1 parent 1:0 protocol ip prio 1 tcindex mask 0xfc
> > shift 2
> 
> flowid/classid here maybe, to get packets to 2:0, though it may work - 
> check.
> 
> > 
> > Main htb qdisc & class
> > tc qdisc add dev $1 parent 1:0 handle 2:0 htb
> > tc class add dev $1 parent 2:0 classid 2:1 htb rate 100Mbit ceil 100Mbit
> 
> 100mbit will be too high if it's 100mbit nic.
> 
> > 
> > EF Class (2:10)
> > tc class add dev $1 parent 2:1 classid 2:10 htb rate 1500Kbit ceil
> > 100Mbit
> > tc qdisc add dev $1 parent 2:10 pfifo limit 5
> > tc filter add dev $1 parent 2:0 protocol ip prio 1 handle 0x2e tcindex
> > classid 2:10 pass_on
> 
> Don't know what pass_on will mean here.
> 
> > 
> > BE Class (2:20)
> > tc class add dev $1 parent 2:1 classid 2:20 htb rate 5Mbit ceil 100Mbit
> > tc qdisc add dev $1 parent 2:20 red limit 60KB min 15KB max 45KB burst
> > 20 avpkt 1000 bandwidth 100Mbit probability 0.4
> > tc filter add dev $1 parent 2:0 protocol ip prio 2 handle 0 tcindex mask
> > 0 classid 2:20 pass_on
> > 
> > Config 2 using PRIO
> > -------------------
> > Main dsmark & classifier
> > tc qdisc add dev $1 handle 1:0 root dsmark indices 64 set_tc_index
> > tc filter add dev $1 parent 1:0 protocol ip prio 1 tcindex mask 0xfc
> > shift 2
> > 
> > Main prio queue
> > tc qdisc add dev $1 parent 1:0 handle 2:0 prio
> > tc qdisc add dev $1 parent 2:1 tbf rate 1.5Mbit burst 1.5kB limit 1.6kB
> 
> Won't hurt if the packets are small voip but TBF has a nasty habit of 
> taking 1 from the burst/mtu you specify so your burst setting may result 
> in packets >1499B getting dropped - tc -s -d qdisc ls dev ... should 
> show what it's using.
> 
> > tc filter add dev $1 parent 2:0 protocol ip prio 1 handle 0x2e tcindex
> > classid 2:1 pass_on
> > 
> > BE class(2:2)
> > tc qdisc add dev $1 parent 2:2 red limit 60KB min 15KB max 45KB burst 20
> > avpkt 1000 bandwidth 100Mbit probability 0.4
> > tc filter add dev $1 parent 2:0 protocol ip prio 2 handle 0 tcindex mask
> > 0 classid 2:2 pass_on
> 
> Without wrapping it with something like htb red won't shape traffic.
> 
> Andy.

_______________________________________________
LARTC mailing list
LARTC@xxxxxxxxxxxxxxx
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux