Hi.
I've run the below script on my servers for a while now and thought I
would submit it for comment or use by others. Also to elicit
improvements.
Basically it creates a number of HTB streams and sub-streams for one
of our 100mb networks.
Logically important is to understand that this is for a gateway
server. ie, it is between the internet and the various LAN's and all
traffic goes via it. Why is this important? Because it does not
police the traffic (ie, drop packets) but queues the traffic both
inbound and outbound on two separate interfaces. As a result, it
makes sure that incoming traffic obeys the configured traffic rules
without policing.
Also note that the script is written for synchronous traffic, where
the incoming and outgoing rates are equal. In other words, it is not
written to handle 30mb download and 1mb upload.
The logical structure is:
.--------------.---------.
| Root | Filters |
'--------------'---------'
|
v
.--------------.
| HTB Limit |-----------------------------------------------.
'--------------' |
| |
v v
.--------------------. .--------------------.
| HTB Stream | | HTB Stream |
| Guaranteed/Ceiling |---------. | Guaranteed/Ceiling |
'--------------------' | '--------------------'
| | |
v v |
.-------------. .-------------. v
| Optionally | | Optionally | .-----------.
| other HTB | | other HTB |-------------------->| ditto.... |
| sub streams | | sub streams | '-----------'
'-------------' '-------------'
|
v
.----------------.
| Prio qdisc |-----------------------.
'----------------' | |
| | |
v v v
.------------. .------------. .------------.
| prio 0 | | prio 1 | | prio 2 |
'------------' '------------' '------------'
| | |
v v v
.-----------. .-----------. .-----------.
| qdisc sfq | | qdisc sfq | | qdisc sfq |
'-----------' '-----------' '-----------'
Textually, there is a main HTB limiting the traffic to 0.5mbit below
the rated line speed (adjustable to suit your config), which is then
broken into HTB streams which can be optionally broken into further
HTB sub-streams.
Finally, the base of each HTB receive a standard prio qdisc and each
of the prio qdisc's receive a sfq qdisc to stop any one system from
hogging all the bandwidth within that stream.
It works very well for us.
Note that as per usual the IP addresses are bogus and that the 'names'
are just invented to protect the innocent.
Here is the script:
#!/bin/bash
# Debugging
#set -x
# Traffic Shaping
# (c) 2012 KCS Total Solutions Ltd
# Released 2013-12-16 AGPL 3.0
# Contact: Ian Macintosh
# email address is ian.macintosh at the domain kcsts.co.uk
# 2012-03-13 - IGM: Initial version
# 2013-02-20 - IGM: Updates for LAN2
# 2013-05-01 - IGM: Added PRIO & SFQ
# 2013-12-16 - IGM: Refactored - changed to variables & loops
# Interfaces - Inside and Outside
IFACE_OUTSIDE=eth0
IFACE_INSIDE=eth1
# Maximum mbit rate on forwarding
MAX_IRATE=99.9
MAX_ORATE=99.9
# mbit Guaranteed Rates (GRate) & Ceiling Rates (CRate)
declare -a Names # Bucket Names (human understanding only)
declare -a GRate # Guaranteed rate
declare -a CRate # Ceiling rate
declare -a SubDv # Is this bucket subdivided?
declare -a FiltI # Filter IP address
declare -a FiltP # Filter Priority - generally /32 IP filters
or dead accurate /XX filters can be prio 1
# but other 'over-broad' filters should be a
lower priority, eg, prio 2
DefaultFilter=11 # Default classify, (Main is #1, and it is
subdivided with sub #1=Firewall
Names=(Main Linus Bill Larry Sergey Peter James )
GRate=(87 2 2 2 2 2 2 )
CRate=(99 20 20 20 20 20 20 )
SubDv=(y n n n n n n )
# The filters to extract put the right IP's into the correct streams
# Note that any subdivided (SubDv="y") stream does not use this filter
# but instead use the filter given in the array SubFI below
FiltI=( \
XXXXXXXXXXXXXXX \
141.0.56.105/32 \
141.0.56.104/29 \
141.0.56.117/32 \
141.0.56.100/32 \
141.0.56.120/32 \
141.0.56.103/32 \
)
# The filter priority. Any overbroad filter should receive a lower
(higher number) priority
FiltP=( \
X \
1 \
2 \
1 \
1 \
1 \
1 \
)
# SubDivided rates (SubD)
declare -a SubDN # SubD Names
declare -a SubDG # SubD Guaranteed rates
declare -a SubDC # SubC Ceiling rates
declare -a SubFi # Subdivided Filters
SubDN=( 'subn=(LAN Backup Web Other )' )
SubDG=( 'subg=(46.5 10 10 20 )' )
SubDC=( 'subc=(99 99 99 99 )' )
# IP filter specs for each class, in sequence
SubFi=( \
'subf=( \
"" \
"101.10.16.99/32" \
"101.10.16.97/32 101.10.16.112/32 111.10.16.113/32 101.10.16.114/32
101.10.16.116/32 101.10.16.118/32 101.10.16.124/32" \
"101.10.16.98/32" \
)' \
)
if [ "$1" = "status" ]; then
echo
echo "INSIDE"
echo "======"
tc -s -d -p -iec qdisc show dev $IFACE_INSIDE
tc -s -d -p -iec class show dev $IFACE_INSIDE
echo
echo "OUTSIDE"
echo "======="
tc -s -d -p -iec qdisc show dev $IFACE_OUTSIDE
tc -s -d -p -iec class show dev $IFACE_OUTSIDE
echo
exit 0
fi
# Start fresh
tc qdisc del dev $IFACE_OUTSIDE root &> /dev/null
tc qdisc del dev $IFACE_INSIDE root &> /dev/null
# If we're stopping, we're done
if [ "$1" = "stop" ]; then
exit
fi
# abbreviations
TCQI="tc qdisc add dev $IFACE_INSIDE"
TCQO="tc qdisc add dev $IFACE_OUTSIDE"
TCCI="tc class add dev $IFACE_INSIDE"
TCCO="tc class add dev $IFACE_OUTSIDE"
TCFI="tc filter add dev $IFACE_INSIDE protocol ip"
TCFO="tc filter add dev $IFACE_OUTSIDE protocol ip"
$TCQI root handle 1: htb default $DefaultFilter
$TCCI parent 1: classid 1:1 htb rate ${MAX_IRATE}mbit
$TCQO root handle 1: htb default $DefaultFilter
$TCCO parent 1: classid 1:1 htb rate ${MAX_ORATE}mbit
for i in $(seq 0 $((${#Names[@]} - 1)) ); do
ClassID=$((i + 1))
#echo "Creating HTB class for ${Names[i]} at rates G/C=${GRate[i]}/${CRate[i]}"
$TCCI parent 1:1 classid 1:${ClassID}0 htb rate ${GRate[i]}mbit ceil
${CRate[i]}mbit
$TCCO parent 1:1 classid 1:${ClassID}0 htb rate ${GRate[i]}mbit ceil
${CRate[i]}mbit
if [ ${SubDv[i]} = "y" ]; then
eval ${SubDN[i]}; eval ${SubDG[i]}; eval ${SubDC[i]}; eval ${SubFi[i]}
for j in $(seq 0 $((${#subn[@]} - 1)) ); do
ClassSuff=$((j + 1))
$TCCI parent 1:${ClassID}0 classid 1:${ClassID}${ClassSuff} htb
rate ${subg[j]}mbit ceil ${subc[j]}mbit
$TCCO parent 1:${ClassID}0 classid 1:${ClassID}${ClassSuff} htb
rate ${subg[j]}mbit ceil ${subc[j]}mbit
for ks in ${subf[j]}; do
if [ ${#ks} -gt 0 ]; then
$TCFI parent 1: prio 1 u32 match ip dst $ks flowid 1:${ClassID}${ClassSuff}
$TCFO parent 1: prio 1 u32 match ip src $ks flowid 1:${ClassID}${ClassSuff}
fi
done
$TCQI parent 1:${ClassID}${ClassSuff} handle ${ClassID}0${ClassSuff}: prio
$TCQO parent 1:${ClassID}${ClassSuff} handle ${ClassID}0${ClassSuff}: prio
for k in $(seq 1 3); do
$TCQI parent ${ClassID}0${ClassSuff}:$k handle
${ClassID}${ClassSuff}$k: sfq perturb 10
$TCQO parent ${ClassID}0${ClassSuff}:$k handle
${ClassID}${ClassSuff}$k: sfq perturb 10
done
done
else
$TCFI parent 1: prio 1 u32 match ip dst ${FiltI[i]} flowid 1:${ClassID}0
$TCFO parent 1: prio 1 u32 match ip src ${FiltI[i]} flowid 1:${ClassID}0
$TCQI parent 1:${ClassID}0 handle ${ClassID}0: prio
$TCQO parent 1:${ClassID}0 handle ${ClassID}0: prio
for j in $(seq 1 3); do
$TCQI parent ${ClassID}0:$j handle ${ClassID}0$j: sfq perturb 10
$TCQO parent ${ClassID}0:$j handle ${ClassID}0$j: sfq perturb 10
done
fi
done
exit 0
Comments, flames, critique and improvements accepted - either to the
list or in the case of flames preferably me directly :-)
Regards,
Ian.
-------------------
Ian Macintosh
Technical Director - KCS Total Solutions Ltd
Tel: +44 (0)1442 251-514 Web: www.kcsts.co.uk
Registered in England and Wales, No. 3792344 at 3 Kensworth Gate,
200-204 High Street South, Dunstable, LU6 3HS
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
