On 07/07/14 10:54, Michal Soltys wrote:
tc qdisc add dev ppp0 root handle 1:0 hfsc stab overhead 40 linklayer atm default 14
tc class add dev ppp0 parent 1:0 classid 1:1 hfsc ls m2 1100000 ul m2 1100000
tc class add dev ppp0 parent 1:1 classid 1:10 hfsc sc m2 100kbit #syn ack rst
tc class add dev ppp0 parent 1:1 classid 1:11 hfsc sc m1 500kbit d 20ms m2 300kbit # Time critical
tc class add dev ppp0 parent 1:1 classid 1:12 hfsc sc m2 200kbit #Interactive
tc class add dev ppp0 parent 1:1 classid 1:13 hfsc sc m2 100kbit #bulk
tc class add dev ppp0 parent 1:1 classid 1:14 hfsc sc m1 100kbit d 20ms m2 300kbit # torrent and not-classified junk
Hi,
Please note that the below relates to the updated traffic classes
detailed within this email...
I have been spending further time testing stuff this morning/afternoon.
I noticed that when download was busy and I started an upload the
download speed was fluctuating up and down in a rhythmic pattern. I
theorise that this is likely caused by the downloads ACK packets (which
are small and categorised the same as the download traffic) not getting
sent. During a download at line rate with no downstream traffic shaping
enabled I see around 350kbit/sec flowing 'into' eth1 on the server.
This will be traffic that is destined for the internet.
I am categorising the traffic as follows:
If a rule is matched we stop processing further rules.
Destination port 80 Packet length 0->512 bytes class 1:12
Destination port 80 Packet length 512+ AND bytes out LESS than 5MB class
1:12
Destination port 80 Packet length 512+ AND bytes out MORE than 5MB class
1:13
The bytes out rules look in conntrack to see how much data has flowed in
that connection since it started.
I have confirmed that packets flowing up relating to this download are
being categorised by the first rule, they are showing as 52 bytes long
and have the ACK flag set.
With downstream traffic management disabled and idle upstream I am
seeing steady line rate transfer on my download... (Output is wget
--progress=dot:mega on a massive file hosted on a fast server.)
24576K ........ ........ ........ ........ ........ ........ 0% 2.03M
53m9s
27648K ........ ........ ........ ........ ........ ........ 0% 2.03M
51m16s
30720K ........ ........ ........ ........ ........ ........ 0% 2.00M
49m46s
33792K ........ ........ ........ ........ ........ ........ 0% 2.03M
48m28s
36864K ........ ........ ........ ........ ........ ........ 0% 2.03M
47m23s
However once I start an sftp upload (which is categorised 1:13) my
download speed becomes eratic with rhythmic slowdowns:
86016K ........ ........ ........ ........ ........ ........ 2% 2.00M
40m20s
89088K ........ ........ ........ ........ ........ ........ 2% 2.05M
40m5s
92160K ........ ........ ........ ........ ........ ........ 2% 2.03M
39m52s
95232K ........ ........ ........ ........ ........ ........ 2% 2.00M
39m41s
98304K ........ ........ ........ ........ ........ ........ 2% 1.21M
40m11s
101376K ........ ........ ........ ........ ........ ........ 2% 1.20M
40m41s
104448K ........ ........ ........ ........ ........ ........ 2% 1.39M
40m55s
107520K ........ ........ ........ ........ ........ ........ 2% 1.66M
40m55s
110592K ........ ........ ........ ........ ........ ........ 2% 1.66M
40m54s
113664K ........ ........ ........ ........ ........ ........ 2% 1.58M
40m57s
116736K ........ ........ ........ ........ ........ ........ 2% 1.75M
40m53s
119808K ........ ........ ........ ........ ........ ........ 2% 1.73M
40m50s
122880K ........ ........ ........ ........ ........ ........ 2% 1.86M
40m42s
125952K ........ ........ ........ ........ ........ ........ 2% 1.94M
40m33s
129024K ........ ........ ........ ........ ........ ........ 3% 1.89M
40m26s
132096K ........ ........ ........ ........ ........ ........ 3% 1.71M
40m24s
135168K ........ ........ ........ ........ ........ ........ 3% 1.70M
40m22s
138240K ........ ........ ........ ........ ........ ........ 3% 1.54M
40m26s
141312K ........ ........ ........ ........ ........ ........ 3% 1.17M
40m47s
144384K ........ ........ ........ ........ ........ ........ 3% 1.21M
41m5s
147456K ........ ........ ........ ........ ........ ........ 3% 1.54M
41m8s
150528K ........ ........ ........ ........ ........ ........ 3% 1.80M
41m2s
153600K ........ ........ ........ ........ ........ ........ 3% 1.77M
40m58s
156672K ........ ........ ........ ........ ........ ........ 3% 1.89M
40m51s
159744K ........ ........ ........ ........ ........ ........ 3% 1.80M
40m46s
162816K ........ ........ ........ ........ ........ ........ 3% 1.62M
40m45s
165888K ........ ........ ........ ........ ........ ........ 3% 1.84M
40m40s
168960K ........ ........ ........ ........ ........ ........ 3% 1.92M
40m32s
172032K ........ ........ ........ ........ ........ ........ 4% 1.71M
40m30s
175104K ........ ........ ........ ........ ........ ........ 4% 1.56M
40m32s
178176K ........ ........ ........ ........ ........ ........ 4% 1.70M
40m29s
181248K ........ ........ ........ ........ ........ ........ 4% 1.31M
40m39s
184320K ........ ........ ........ ........ ........ ........ 4% 1.21M
40m53s
187392K ........ ........ ........ ........ ........ ........ 4% 1.26M
41m4s
190464K ........ ........ ........ ........ ........ ........ 4% 1.74M
41m0s
193536K ........ ........ ........ ........ ........ ........ 4% 1.75M
40m56s
196608K ........ ........ ........ ........ ........ ........ 4% 1.87M
40m50s
199680K ........ ........ ........ ........ ........ ........ 4% 1.91M
40m43s
202752K ........ ........ ........ ........ ........ ........ 4% 1.89M
40m37s
205824K ........ ........ ........ ........ ........ ........ 4% 1.92M
40m30s
208896K ........ ........ ........ ........ ........ ........ 4% 1.96M
40m23s
Here is the my tc structure
#QoS for Upload
tc qdisc del dev ppp0 root
tc qdisc add dev ppp0 stab overhead 42 linklayer atm mtu 1458 root
handle 1:0 hfsc default 14
tc class add dev ppp0 parent 1:0 classid 1:1 hfsc sc rate 98mbit
tc class add dev ppp0 parent 1:1 classid 1:2 hfsc ls m2 1100kbit ul m2
1100kbit
tc class add dev ppp0 parent 1:2 classid 1:11 hfsc sc m1 600kbit d 20ms
m2 400kbit # Time critical
tc class add dev ppp0 parent 1:2 classid 1:12 hfsc sc m2 300kbit
#Interactive
tc class add dev ppp0 parent 1:2 classid 1:13 hfsc sc m2 75kbit #bulk
tc class add dev ppp0 parent 1:1 classid 1:14 hfsc sc rate 98mbit
tc qdisc add dev ppp0 parent 1:11 handle 11: sfq (have also tried pfifo)
tc qdisc add dev ppp0 parent 1:12 handle 12: sfq limit 2 (have also
tried "pfifo limit 2" "bfifo limit 2916b")
tc qdisc add dev ppp0 parent 1:13 handle 13: sfq limit 2 (have also
tried "pfifo limit 2" "bfifo limit 2916b")
tc qdisc add dev ppp0 parent 1:14 handle 14: pfifo
tc filter add dev ppp0 parent 1:0 protocol ip prio 2 handle 11 fw flowid
1:11
tc filter add dev ppp0 parent 1:0 protocol ip prio 3 handle 12 fw flowid
1:12
tc filter add dev ppp0 parent 1:0 protocol ip prio 4 handle 13 fw flowid
1:13
My method for devising the above was:
Total rt traffic: 975kbit
Time critical (voip, ping etc) guaranteed 400kbit
Left over bandwidth shared by ratio 3:1. EG voip is using 400kbit this
means there is 700kbit left. If all classes were saturated this would
mean interactive got 466kbit and bulk got 233kbit.
My testing has around 64bytes/sec hitting the time critical queue (just
a ping packet every seconnd). This should mean that an http download
sending 350kbit of ACKs will be able to send all of its packets because
it should be allocated 3/4 of the remaining upload (777kbit) with the
remaining 259kbit used by the bulk upload...
However I see more traffic dropping in my queue 1:12:
class hfsc 1:11 parent 1:2 leaf 11: sc m1 600000bit d 20.0ms m2 400000bit
Sent 703310 bytes 4124 pkt (dropped 56, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
period 3987 work 703310 bytes rtwork 618669 bytes level 0
class hfsc 1:12 parent 1:2 leaf 13: sc m1 0bit d 0us m2 75000bit
Sent 5553128 bytes 5024 pkt (dropped 452, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
period 4954 work 5553128 bytes rtwork 915469 bytes level 0
class hfsc 1:13 parent 1:2 leaf 12: sc m1 0bit d 0us m2 300000bit
Sent 6719923 bytes 60826 pkt (dropped 1176, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
period 33210 work 6719923 bytes rtwork 3253617 bytes level 0
qdisc pfifo 11: parent 1:11 limit 3p
Sent 736170 bytes 4324 pkt (dropped 56, overlimits 0 requeues 0)
qdisc bfifo 12: parent 1:12 limit 2916b
Sent 6733385 bytes 60906 pkt (dropped 1176, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
qdisc bfifo 13: parent 1:13 limit 2916b
Sent 5554877 bytes 5030 pkt (dropped 452, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
I cant understand why queue 12 is dropping packets and not getting 3/4
of the bandwidth share? Maybe my STAB is still incorrect and all the
small packets are throwing out the calculations?
Other information you might need:
I chose limit 2 based upon my desire to have no more than 60ms jitter.
(1458/110000)*4=0.05301818 or 53ms.
Queue 1:11 is for time critical traffic. I am categorising small tcp
connection related packets (see below), VoIP, online games, ping and DNS
here.
Queue 1:12 is for interactive traffic. I categorise short ssh packets,
short dport 80 short dport 443, RDP, email, and web flows less than 5MB
but large packets here.
Queue 1:13 is for bulk traffic. I categorise web flows over 5MB and
everything else else into this queue.
TCP related packets hitting queue 1:11 are categorised by the following
iptables rule:
iptables -t mangle -A ULQOS -p tcp --syn -m length --length 40:68 -j
MARK --set-mark 11
iptables -t mangle -A ULQOS -p tcp --tcp-flags ALL SYN,ACK -m length
--length 40:68 -j MARK --set-mark 11
iptables -t mangle -A ULQOS -p tcp --tcp-flags ALL RST -j MARK --set-mark 11
iptables -t mangle -A ULQOS -p tcp --tcp-flags ALL ACK,RST -j MARK
--set-mark 11
iptables -t mangle -A ULQOS -p tcp --tcp-flags ALL ACK,FIN -j MARK
--set-mark 11
Thanks again for your patience and support. Sorry for continuing to be a
bit of a dunce!
Alan
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
