Hello Stef, > The other problem is the parent rate and ceil. I did some test and I > have strange results. I still have to find a way to find out how you > can configure the parent rate and ceil to get a good setup. During my search I did some quick tests based on what you and others had written. Those tests revealed that I can setup things the way I want. I used following commands to set everything up. tc qdisc add dev lo root handle 1:0 htb default 4 tc class add dev lo classid 1:1 parent 1:0 htb rate 512kbit ceil 512kbit tc class add dev lo classid 1:2 parent 1:1 htb rate 8kbit ceil 128kbit tc class add dev lo classid 1:3 parent 1:1 htb rate 8kbit ceil 128kbit #class used for unclassified traffic tc class add dev lo classid 1:4 parent 1:1 htb rate 8kbit ceil 128kbit #children of class B #interactive traffic tc class add dev lo classid 1:21 parent 1:2 htb rate 4kbit ceil 64kbit #web traffic tc class add dev lo classid 1:22 parent 1:2 htb rate 3kbit ceil 128kbit #other traffic tc class add dev lo classid 1:23 parent 1:2 htb rate 1kbit ceil 32kbit #children of class C tc class add dev lo classid 1:31 parent 1:3 htb rate 4kbit ceil 64kbit tc class add dev lo classid 1:32 parent 1:3 htb rate 3kbit ceil 128kbit tc class add dev lo classid 1:33 parent 1:3 htb rate 1kbit ceil 32kbit tc filter add dev lo parent 1:0 prio 10 protocol ip u32 divisor 256 tc filter add dev lo parent 1:0 prio 10 u32 match ip dport 5021 0xffff flowid 1:21 tc filter add dev lo parent 1:0 prio 10 u32 match ip dport 5022 0xffff flowid 1:22 tc filter add dev lo parent 1:0 prio 10 u32 match ip dport 5023 0xffff flowid 1:23 tc filter add dev lo parent 1:0 prio 10 u32 match ip dport 5031 0xffff flowid 1:31 tc filter add dev lo parent 1:0 prio 10 u32 match ip dport 5032 0xffff flowid 1:32 tc filter add dev lo parent 1:0 prio 10 u32 match ip dport 5033 0xffff flowid 1:33 MTU of loopback device is modified and set to 1500 bytes. Measurements results are (a little modified) output from command "showtraf -i lo -n". Traffic was generated by command "ttcp -s -r 127.0.0.1 -p portno -v -f k" and "ttcp -s -t 127.0.0.1 -p portno -v -f k" where portno is 5021,5022,5023,5031,5032,5033. Here goes the results. >From Address To Address Proto Bytes CPS ====================================================================== 127.0.0.1..3143 127.0.0.1..5022 tcp 9864112 6300 127.0.0.1..3142 127.0.0.1..5021 tcp 9307500 6300 127.0.0.1..3145 127.0.0.1..5031 tcp 8332612 6300 127.0.0.1..3147 127.0.0.1..5032 tcp 7641112 6000 127.0.0.1..3144 127.0.0.1..5023 tcp 6237112 3900 127.0.0.1..3148 127.0.0.1..5033 tcp 5326612 3900 It is clear that ceiling of parent class (2 and 3) is respected. The sum of 22+23+24 and 31+32+33 is around 128kbit/s. Bandwidth inside class 2 and 3 is divided according (proportional) to rates and never exceeds leaf class ceil. During the tests I discovered that in case of root class (1:0 in my example) only rate matters not ceil. I accidentally changed 1:2 and 1:3 to root classes and then 22,23,24 were limited to 8kbit/s. Then I simulated congestion on main link by issuing tc class change dev lo classid 1:1 parent 1:0 htb rate 192kbit ceil 192kbit >From Address To Address Proto Bytes CPS ====================================================================== 127.0.0.1..3143 127.0.0.1..5022 tcp 14361112 4200 127.0.0.1..3142 127.0.0.1..5021 tcp 13909500 3900 127.0.0.1..3145 127.0.0.1..5031 tcp 12940612 4200 127.0.0.1..3147 127.0.0.1..5032 tcp 12130612 3900 127.0.0.1..3144 127.0.0.1..5023 tcp 9298612 3600 127.0.0.1..3148 127.0.0.1..5033 tcp 8386612 3600 And again, everything worked as expected. Bandwidth inside 1:1 was divided equaly between 1:2 and 1:3. It was time to change rate of 1:2 to 64kbit (change to 16kbit/s hadn't introduced any significant effects). tc class change dev lo classid 1:2 parent 1:1 htb rate 64kbit ceil 128kbit Results: (i will refer to them further as Exapmle A results) >From Address To Address Proto Bytes CPS ====================================================================== 127.0.0.1..3203 127.0.0.1..5021 tcp 6640612 5400 127.0.0.1..3204 127.0.0.1..5022 tcp 5955112 5100 127.0.0.1..3205 127.0.0.1..5023 tcp 3372612 4200 127.0.0.1..3206 127.0.0.1..5031 tcp 3309112 3000 127.0.0.1..3207 127.0.0.1..5032 tcp 2487112 3000 127.0.0.1..3209 127.0.0.1..5033 tcp 1716112 2700 Hm, strange - it seems bandwidth is divided about 2:1. tc class show dev lo gives us: class htb 1:3 parent 1:1 prio 0 rate 8Kbit ceil 128Kbit burst 1609b/8 mpu 0b cburst 1762b/8 mpu 0b quantum 102 level 2 Sent 61860798 bytes 40875 pkts (dropped 0, overlimits 0) rate 8786bps 5pps lended: 849 borrowed: 37041 giants: 0 injects: 0 tokens: -4918741 ctokens: -25831 class htb 1:2 parent 1:1 prio 0 rate 64Kbit ceil 128Kbit burst 1680b/8 mpu 0b cburst 1762b/8 mpu 0b quantum 819 level 2 Sent 74831096 bytes 49440 pkts (dropped 0, overlimits 0) rate 14859bps 9pps lended: 11565 borrowed: 34638 giants: 0 injects: 0 tokens: -23468 ctokens: -15768 Only things that differ are quantum and rate, but they differ in propotion 1:8. After changing class 1:0 rate and bandwidth to 256kbits the difference is smaller. I doubled rate of 1:2 and do the same to 1:21,22,23. Results: (i will refer to them further as Exapmle B results) >From Address To Address Proto Bytes CPS ====================================================================== 127.0.0.1..3216 127.0.0.1..5021 tcp 4782112 4500 127.0.0.1..3217 127.0.0.1..5022 tcp 4504612 3900 127.0.0.1..3219 127.0.0.1..5031 tcp 4143112 3900 127.0.0.1..3220 127.0.0.1..5032 tcp 4078612 3600 127.0.0.1..3218 127.0.0.1..5023 tcp 3412612 3600 127.0.0.1..3221 127.0.0.1..5033 tcp 3304612 3900 Well..... I hoped 1:2 will be given more bandwidth:) .... In my opinion when prios are equal every leaf at the same level is given bandwith equal to rate, then bandwidth equal to the rate of parent class is divided between all children equally. If some children request less (due to ceil limitations or real needs), available bandwidth is distributed equally between other children. And then available bandwidth of grandparent is divided between children equally (ceil of parent is respected). Let's analyze Example A to make things clear: 192kbit/s = 24000 CPS .... showtraf always show sum of 23400 (it is possible that 192kbit/s is layer 2 value, showtraf takes its measurements at level 4). Step 1. 1:21 is given 500 CPS 1:22 is given 400 CPS 1:23 is given 100 CPS 1:31 is given 500 CPS 1:32 is given 400 CPS 1:33 is given 100 CPS Step 1 SUM=2000CPS Step2. 1:2 rate of 8000 CPS is divided between 21,22,23 equally. 1:21 is given ca 2650 CPS 1:22 is given ca 2650 CPS 1:23 is given ca 2650 CPS 1:3 rate of 1000 CPS is divided between 31,32,33 equally. 1:31 is given ca 350 CPS 1:32 is given ca 350 CPS 1:33 is given ca 350 CPS Step 2 SUM==9000 CPS None of leaf classes is currently limited by ceil or parent ceil. Step 3. There is 13000 CPS to share between six classes Each class gets 13000/6=2150, but then class 1:23 would be transmitting over ceil so class 1:23 gets ca 1450 and is sending packets at speed 4200 CPS (this value oscilates around 4000 CPS). 13000-1450=11550 11850 is divided into five parts (five classes), each class gets 2300 Final results are: 1:21 should be transmitting at rate 500+2650+2300=5450 (real time measurement oscilates around this value, Example A shows value of 5400) 1:22 should be transmitting at rate 400+2650+2300=5350 (show traf shows 5100) 1:23 sould be transmitting at 4000 (limited by ceil, it is sending packets at speed of 4200) 1:31 500+350+2300=3150 (showtraf shows 3000) 1:32 400+250+2300=3050 (showtraf shows 3000) 1:33 100+350+2300=2750 (showtraf shows 2700) As for Example B - calculations allocate bandwidth in following fashion: 1:21 - 4650 CPS 1:22 - 4450 CPS 1:23 - 3850 CPS 1:31 - 3850 CPS 1:32 - 3750 CPS 1:33 - 3450 CPS which is close to reality (values from show traf oscilate around values about 3-5% lower). I think if I use prios child classes with lower prio value will be offered bandwidth first. I will do some future research in this area. I am thinking about analyzing a little bigger, real life example with 4 levels of classes, different prios etc.. I hope somebody except me benefits from this post. Robert Kryczało
