I think I will migrate the ingress traffic to IFB as it mentioned in other thread and drop the egress shaping at the internal interface. Didn't know that IMQ was a thing of past and IFB is a successor to it. There's not many documentations about IMQ and IFB on the internet. Horace Ng - Internet Solutions Limited E-mail: horace@xxxxxxxxx Tel: +852 27109880 Fax: +852 27704631 ----- Original Message ----- From: "Horace" <horace@xxxxxxxxxxxxxxx> To: "Remy Mudingay" <remy.mudingay@xxxxxxxxx> Cc: lartc@xxxxxxxxxxxxxxx Sent: Thursday, August 15, 2013 3:39:22 PM Subject: Re: Not hitting rated speed on HTB downstream Hi Mudingay, My actual maximum throughput is 1Gbps, I just merely try to chunk out 10mbps for one of my client. As I mentioned before, if I only modify the ceil value to 30mbit for 1:2 class, it can actually attain full 10mbit for that link, nothing else need to be modified. One thing that I noticed is that the 1:2 class sometimes got a negative token sum when transferring at 4mbps average, which from my understanding, it is either bursting too much (not really, the physical line speed is only 10mbps), or somehow the token is just not adding fast enough for 10mbit!! (clocksource problem? tried hpet already) Horace Ng - Internet Solutions Limited E-mail: horace@xxxxxxxxx Tel: +852 27109880 Fax: +852 27704631 ----- Original Message ----- From: "Remy Mudingay" <remy.mudingay@xxxxxxxxx> To: "Horace Ng" <horace@xxxxxxxxx> Cc: lartc@xxxxxxxxxxxxxxx Sent: Thursday, August 15, 2013 2:23:12 PM Subject: Re: Not hitting rated speed on HTB downstream HI Horace, Correct me if I miss understood your network topology - Your maximum download rate should be set to 10mbit because that is your limit. Therefore your shaping rules for your default class should also reflect this by setting the ceil to 10mbit also. Your class statistics show that 4270Kbit (4.3mbits) are being shaped by classid 1:2, classid 1:3 has 672bit and the default classid (1:4) has 4647Kbit(4.6mbit). So roughly half of your network traffic if going into your default class (1:4) and since your default class has a guaranteed rate of 1Gbit... If you choose to go the HFSC route, it is important to note that HFSC is more complicated to setup than HTB and it will for example drop any packet which has not been explicitly classified, whereas HTB will either bypass (direct_packet_stat) or shape these packets in the default class. I highly recommend that you identify the following : a) what is going through your default class (classid 1:4)? #This can be achieved by using the mirred action filter. firstly, modify this line tc qdisc add dev bond0.10 parent 1:4 pfifo to the following tc qdisc add dev bond0.10 parent 1:4 handly 4:1 pfifo #Next add a dummy interface modprobe dummy #Then add a classification rule tc filter add dev bon0.10 prio 1 protocol all parent 4:1 action mirred egress mirror dev dummy0 #run tcpdump tcpdump -i dummy0 Now you have identified the network that is using up half your of your total bandwidth. b) The rate sum of classid 1:2, 1:3 and 1:4 should not be greater than 10mbit. I hope this helps. Remy On 15 August 2013 03:59, Horace <horace@xxxxxxxxxxxxxxx> wrote: > I've tried to modify classid 1:4 to 500mbit, seems no difference. What do you mean by lower the ceil value? > > Horace Ng > - > Internet Solutions Limited > E-mail: horace@xxxxxxxxx > Tel: +852 27109880 > Fax: +852 27704631 > > ----- Original Message ----- > From: "Remy Mudingay" <remy.mudingay@xxxxxxxxx> > To: "Horace Ng" <horace@xxxxxxxxx> > Cc: lartc@xxxxxxxxxxxxxxx > Sent: Tuesday, August 13, 2013 11:57:29 PM > Subject: Re: Not hitting rated speed on HTB downstream > > Hi, > > Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4. > Secondly, I recommend lowering/setting the ceil value to 10mbit (or slightly lower). > > > > > Sent from my iPhone > > On Aug 13, 2013, at 6:35, Horace <horace@xxxxxxxxxxxxxxx> wrote: > >> Hi, >> >> Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients >> <-- 10mbit link ---> clients (2ms latency) >> >> >> I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link). >> >> I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables. >> >> Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference. >> >> Any suggestion how to solve this problem? >> >> Here's tc-htb config >> ==================== >> tc qdisc del dev bond0.10 root >> tc qdisc add dev bond0.10 root handle 1: htb default 4 >> tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1 >> tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1 >> tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1 >> tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2 >> >> tc qdisc add dev bond0.10 parent 1:2 pfifo >> tc qdisc add dev bond0.10 parent 1:3 pfifo >> tc qdisc add dev bond0.10 parent 1:4 pfifo >> >> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \ >> match ip dst 1.1.1.8/29 flowid 1:2 >> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \ >> match ip dst 1.1.1.81 flowid 1:3 >> >> Here's the stats >> ================ >> class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7 >> Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0) >> rate 8919Kbit 1633pps backlog 0b 0p requeues 0 >> lended: 0 borrowed: 0 giants: 0 >> tokens: 16 ctokens: 16 >> >> class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 >> Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0) >> rate 4270Kbit 398pps backlog 0b 1p requeues 0 >> lended: 16930 borrowed: 0 giants: 0 >> tokens: -18459 ctokens: -18459 <=============== is it not supposed to be negative when the rate is not more than 10mbit? >> >> class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 >> Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0) >> rate 672bit 0pps backlog 0b 0p requeues 0 >> lended: 61 borrowed: 0 giants: 0 >> tokens: 16610 ctokens: 16610 >> >> class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0 >> Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0) >> rate 4647Kbit 1235pps backlog 0b 0p requeues 0 >> lended: 97706 borrowed: 0 giants: 0 >> tokens: 187 ctokens: 187 >> >> >> >> Horace Ng >> -- >> To unsubscribe from this list: send the line "unsubscribe lartc" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe lartc" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
