Configuring a QoS Box + Cliente Bandwidth Control

Nataniel Klug <nata@xxxxxxxxxxxx> · Wed, 06 Dec 2006 11:23:11 -0200

   Hello all,

   I am trying to configure a linux box to make some QoS into my 
netowork and, at the same box, control my clients bandwidth. I have this 
classes created:

----------------------------------------------------------------
UP="eth0"               # wan infocontabil
DL01="eth2"             # lan clientes

$TC qdisc del dev $DL01 root    2> /dev/null > /dev/null
$TC qdisc del dev $DL01 ingress 2> /dev/null > /dev/null
$TC qdisc del dev $UP root    2> /dev/null > /dev/null
$TC qdisc del dev $UP ingress 2> /dev/null > /dev/null

$TC qdisc add dev $DL01 root handle 1: htb default 40

CLASS="/sbin/tc class add dev $DL01 parent"
$CLASS 1: classid 1:1 htb rate 100Mbit
$CLASS 1:1 classid 1:5 htb rate 100Mbit ceil 100Mbit
$CLASS 1: classid 1:2 htb rate 972Kbit
$CLASS 1:2 classid 1:10 htb rate 128Kbit ceil 256Kbit prio 0
$CLASS 1:2 classid 1:20 htb rate 512Kbit ceil 768Kbit prio 0
$CLASS 1:2 classid 1:30 htb rate 128Kbit ceil 512Kbit prio 1
$CLASS 1:2 classid 1:40 htb rate 204Kbit ceil 512Kbit
----------------------------------------------------------------

   Here, as you can see, I made some rules to control my network. I 
have a class 1:1 that serves only inside my network, so this is not 
limited. I just use this option for some IPs that belongs to my own 
phisical network.

   This is working fine as a QoS becouse I send my traffic as follow:

CLASS 1:10 --> interactive (ssh, telnet)
CLASS 1:20 --> http and https
CLASS 1:30 --> pop, smtp and ftp
CLASS 1:40 --> all the rest

   This is the way my network work better.

   Now my problem is: I have a bunch of clients direct connect into 
eth2 device and I need that, this clients, have some bandwidht control. 
Consider this:

Client IP range: 192.168.0.0/24
Download band: 32 Kbit for each IP

   So I made a script just like this:

-----------------------------
DL="eth2"
CONT="99"

for i in `cat /etc/firewall/qos/hosts.32k`
do
       CONT=`expr $CONT + 1`
       $TC class add dev $DL parent 1:2 classid 1:${CONT} htb rate 
32Kbit ceil 32Kbit
       $TC filter add dev $DL parent 1:0 protocol ip prio 1 u32 match 
ip dst ${i}/32 flowid 1:${CONT}
done
-----------------------------

   I put thi just after the CLASS stuff.

   Now my clients are all full controlled, but my QoS do not work.

   There is some way to make this happens?

PS.: In the end of this email is my full QOS script.

Att,

Nataniel Klug

--------------- start - qos.sh ---------------
#!/bin/sh
#------
# Script de QoS Cyber Nett
#------
# Nataniel Klug
# suporte@xxxxxxxxxxxx
#------

TC="/sbin/tc"
IPT="/usr/local/sbin/iptables"
DIR="/etc/firewall/qos"

UP="eth0"               # wan infocontabil
DL01="eth2"             # lan clientes
DL02="eth3"             # lan infocontabil

#-----
# Limpando iptables
# Aplicando save as marcas (final de cada INTERFACE)
#-----
$IPT -t mangle -F
$IPT -t mangle -X
$IPT -t mangle -A PREROUTING -p tcp -j CONNMARK --restore-mark
$IPT -t mangle -A PREROUTING -p tcp -m mark ! --mark 0 -j ACCEPT

# marcando pacotes
# referente ao P2P
P2PMARK="20"
$IPT -t mangle -A PREROUTING -p tcp -m ipp2p --ipp2p -j MARK --set-mark 
$P2PMARK
$IPT -t mangle -A PREROUTING -p udp -m ipp2p --ipp2p -j MARK --set-mark 
$P2PMARK
$IPT -t mangle -A PREROUTING -p tcp -m ipp2p --ipp2p -j MARK --set-mark 
$P2PMARK
$IPT -t mangle -A PREROUTING -p udp -m ipp2p --ipp2p -j MARK --set-mark 
$P2PMARK
# referente ao skype
SKYPEMARK="21"
$IPT -t mangle -A PREROUTING -p tcp -m layer7 --l7proto skypetoskype -j 
MARK --set-mark $SKYPEMARK
$IPT -t mangle -A PREROUTING -p tcp -m layer7 --l7proto skypeout -j MARK 
--set-mark $SKYPEMARK
$IPT -t mangle -A PREROUTING -p udp -m layer7 --l7proto skypetoskype -j 
MARK --set-mark $SKYPEMARK
$IPT -t mangle -A PREROUTING -p udp -m layer7 --l7proto skypeout -j MARK 
--set-mark $SKYPEMARK
# referente ao msn
MSN="22"
$IPT -t mangle -A PREROUTING -p all -m layer7 --l7proto msnmessenger -j 
MARK --set-mark $MSN
# referente ao ssh
SSH="23"
$IPT -t mangle -A PREROUTING -p all -m layer7 --l7proto ssh -j MARK 
--set-mark $SSH

#----
# SALVANDO MARCAS DO IPTABLES
#----
$IPT -t mangle -A PREROUTING -j CONNMARK --save-mark

#------
# Apagando regras antigas de QoS
#------
$TC qdisc del dev $DL01 root    2> /dev/null > /dev/null
$TC qdisc del dev $DL01 ingress 2> /dev/null > /dev/null
$TC qdisc del dev $DL02 root    2> /dev/null > /dev/null
$TC qdisc del dev $DL02 ingress 2> /dev/null > /dev/null
$TC qdisc del dev $UP root    2> /dev/null > /dev/null
$TC qdisc del dev $UP ingress 2> /dev/null > /dev/null

#------
# CRIANDRO REGRAS DE DOWNLOAD
#------
#=========#
# IF ETH3 #
#  $DL02  #
#=========#
$TC qdisc add dev $DL02 root handle 1: htb default 5

# criandro classes
CLASS="/sbin/tc class add dev $DL02 parent"

# classe de comunicacao com a rede Infocontabil
$CLASS 1: classid 1:1 htb rate 100Mbit
$CLASS 1:1 classid 1:5 htb rate 100Mbit ceil 100Mbit

# classe de redes externas
#$CLASS 1: classid 1:2 htb rate 512Kbit
#$CLASS 1:2 classid 1:10 htb rate 128Kbit ceil 256Kbit prio 0
#$CLASS 1:2 classid 1:20 htb rate 256Kbit ceil 512Kbit prio 0
#$CLASS 1:2 classid 1:30 htb rate 32Kbit ceil 128Kbit prio 1
#$CLASS 1:2 classid 1:40 htb rate 64Kbit ceil 128Kbit

# criandro a fair queue
QDISC="/sbin/tc qdisc add dev $DL02 parent"
$QDISC 1:5 handle 5: sfq perturb 10
#$QDISC 1:10 handle 10: sfq perturb 10
#$QDISC 1:20 handle 20: sfq perturb 10
#$QDISC 1:30 handle 30: sfq perturb 10

# criandro filtros
FILTER="/sbin/tc filter add dev $DL02 parent 1:0 protocol ip"

# regras para servidores e redes da CNett
for i in `cat $DIR/infocontabil.network | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip src $i flowid 1:5
done

# regras para a classe 1:10
# trafego interativo
# PROTOCOLOS
#$FILTER prio 1 u32 match ip protocol 1 0xff flowid 1:10

# PORTAS
#for i in `cat $DIR/prio0.src.ports | awk '{print $2}'`
#do
#        $FILTER prio 1 u32 match ip sport $i 0xffff flowid 1:10
#done

# MARCACAO DE PACOTES
#$IPT -t mangle -A POSTROUTING -o $DL02 -m mark --mark $SKYPEMARK -j 
CLASSIFY --set-class 1:10
#$IPT -t mangle -A POSTROUTING -o $DL02 -m mark --mark $MSN -j CLASSIFY 
--set-class 1:10
#$IPT -t mangle -A POSTROUTING -o $DL02 -m mark --mark $SSH -j CLASSIFY 
--set-class 1:10

# regras para a classe 1:20
# trafego de disponibilidade
# PORTAS
#for i in `cat $DIR/prio1.src.ports | awk '{print $2}'`
#do
#        $FILTER prio 1 u32 match ip sport $i 0xffff flowid 1:20
#done

# regras para a classe 1:30
# trafego de disponibilidade
# PORTAS
#for i in `cat $DIR/prio2.src.ports | awk '{print $2}'`
#do
#        $FILTER prio 1 u32 match ip sport $i 0xffff flowid 1:30
#done

#=========#
# IF ETH2 #
#  $DL01  #
#=========#
$TC qdisc add dev $DL01 root handle 1: htb default 40

# criandro classes
CLASS="/sbin/tc class add dev $DL01 parent"

# classe de comunicacao com a rede Infocontabil
$CLASS 1: classid 1:1 htb rate 100Mbit
$CLASS 1:1 classid 1:5 htb rate 100Mbit ceil 100Mbit

# classe de redes externas
$CLASS 1: classid 1:2 htb rate 972Kbit
$CLASS 1:2 classid 1:10 htb rate 128Kbit ceil 256Kbit prio 0
$CLASS 1:2 classid 1:20 htb rate 512Kbit ceil 768Kbit prio 0
$CLASS 1:2 classid 1:30 htb rate 128Kbit ceil 512Kbit prio 1
$CLASS 1:2 classid 1:40 htb rate 204Kbit ceil 512Kbit

#****
# ADICIONA REGRAS DE CONTROLE DE BANDA
# DOWNLOAD
$DIR/banda.dl

# classe p2p
#$CLASS 1: classid 1:3 htb rate 512Kbit
#$CLASS 1:3 classid 1:45 htb rate 512Kbit ceil 512Kbit

# criandro a fair queue
QDISC="/sbin/tc qdisc add dev $DL01 parent"
#$QDISC 1:5 handle 5: sfq perturb 10
$QDISC 1:10 handle 10: sfq perturb 10
$QDISC 1:20 handle 20: sfq perturb 10
$QDISC 1:30 handle 30: sfq perturb 10

# criandro filtros
FILTER="/sbin/tc filter add dev $DL01 parent 1:0 protocol ip"

# regras para servidores e redes da CNett
for i in `cat $DIR/infocontabil.network | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip src $i flowid 1:5
done

# regras para a classe 1:10
# trafego interativo
# PROTOCOLOS
$FILTER prio 1 u32 match ip protocol 1 0xff flowid 1:10

# PORTAS
for i in `cat $DIR/prio0.src.ports | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip sport $i 0xffff flowid 1:10
done

# MARCACAO DE PACOTES
$IPT -t mangle -A POSTROUTING -o $DL01 -m mark --mark $SKYPEMARK -j 
CLASSIFY --set-class 1:10
$IPT -t mangle -A POSTROUTING -o $DL01 -m mark --mark $MSN -j CLASSIFY 
--set-class 1:10
$IPT -t mangle -A POSTROUTING -o $DL01 -m mark --mark $SSH -j CLASSIFY 
--set-class 1:10

# regras para a classe 1:20
# trafego de disponibilidade
# PORTAS
for i in `cat $DIR/prio1.src.ports | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip sport $i 0xffff flowid 1:20
done

# regras para a classe 1:30
# trafego de disponibilidade
# PORTAS
for i in `cat $DIR/prio2.src.ports | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip sport $i 0xffff flowid 1:30
done

# regras para a classe 1:45
# trafego ruim
# MARCACAO DE PACOTES
$IPT -t mangle -A POSTROUTING -o $DL01 -m mark --mark $P2PMARK -j ACCEPT

#------
# CRIANDO REGRAS DE UPLOAD
#------
#=========#
# IF ETH0 #
#   $UP   #
#=========#
$TC qdisc add dev $UP root handle 1: htb default 40

# criandro classes
CLASS="/sbin/tc class add dev $UP parent"

# classe de comunicacao com a rede Infocontabil
$CLASS 1: classid 1:1 htb rate 100Mbit
$CLASS 1:1 classid 1:5 htb rate 100Mbit ceil 100Mbit

# classe de redes externas
$CLASS 1: classid 1:2 htb rate 972Kbit
$CLASS 1:2 classid 1:10 htb rate 128Kbit ceil 256Kbit prio 0
$CLASS 1:2 classid 1:20 htb rate 512Kbit ceil 768Kbit prio 0
$CLASS 1:2 classid 1:30 htb rate 128Kbit ceil 512Kbit prio 1
$CLASS 1:2 classid 1:40 htb rate 204Kbit ceil 512Kbit

#****
# ADICIONA REGRAS DE CONTROLE DE BANDA
# UPLOAD
$DIR/banda.up

# classe p2p
#$CLASS 1: classid 1:3 htb rate 512Kbit
#$CLASS 1:3 classid 1:45 htb rate 512Kbit ceil 512Kbit

# criandro a fair queue
QDISC="/sbin/tc qdisc add dev $UP parent"
#$QDISC 1:5 handle 5: sfq perturb 10
$QDISC 1:10 handle 10: sfq perturb 10
$QDISC 1:20 handle 20: sfq perturb 10
$QDISC 1:30 handle 30: sfq perturb 10

# criandro filtros
FILTER="/sbin/tc filter add dev $UP parent 1:0 protocol ip"

# regras para servidores e redes da Infocontabil
for i in `cat $DIR/infocontabil.network | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip dst $i flowid 1:5
done

# regras para a classe 1:10
# trafego interativo
# PROTOCOLOS
$FILTER prio 1 u32 match ip protocol 1 0xff flowid 1:10

# PORTAS
for i in `cat $DIR/prio0.src.ports | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip dport $i 0xffff flowid 1:10
done

# MARCACAO DE PACOTES
$IPT -t mangle -A POSTROUTING -o $UP -m mark --mark $SKYPEMARK -j 
CLASSIFY --set-class 1:10
$IPT -t mangle -A POSTROUTING -o $UP -m mark --mark $MSN -j CLASSIFY 
--set-class 1:10
$IPT -t mangle -A POSTROUTING -o $UP -m mark --mark $SSH -j CLASSIFY 
--set-class 1:10

# regras para a classe 1:20
# trafego de disponibilidade
# PORTAS
for i in `cat $DIR/prio1.src.ports | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip dport $i 0xffff flowid 1:20
done

# regras para a classe 1:30
# trafego de disponibilidade
# PORTAS
for i in `cat $DIR/prio2.src.ports | awk '{print $2}'`
do
       $FILTER prio 1 u32 match ip dport $i 0xffff flowid 1:30
done

# regras para a classe 1:45
# trafego ruim
# MARCACAO DE PACOTES
$IPT -t mangle -A POSTROUTING -o $UP -m mark --mark $P2PMARK -j ACCEPT
--------------------- end - qos.sh ----------------------

--------------------- start - banda.dl --------------------
#!/bin/sh
#------
# Nataniel Klug
# suporte@xxxxxxxxxxxx
#------
TC="/sbin/tc"
IPT="/usr/local/sbin/iptables"

DL="eth2"

CONT="99"

#****
# clientes 32k
for i in `cat /etc/firewall/qos/hosts.32k`
do
       CONT=`expr $CONT + 1`
       $TC class add dev $DL parent 1:2 classid 1:${CONT} htb rate 
32Kbit ceil 32Kbit
       $TC filter add dev $DL parent 1:0 protocol ip prio 1 u32 match 
ip dst ${i}/32 flowid 1:${CONT}
done
---------------------- end - banda.dl -------------------

----------------------- start - banda.up ------------------
#!/bin/sh
#------
# Nataniel Klug
# suporte@xxxxxxxxxxxx
#------
TC="/sbin/tc"
IPT="/usr/local/sbin/iptables"

UP="eth0"

CONT="99"

#****
# clientes 32k
for i in `cat /etc/firewall/qos/hosts.32k`
do
       CONT=`expr $CONT + 1`
       $TC class add dev $UP parent 1:2 classid 1:${CONT} htb rate 
16Kbit ceil 16Kbit
       $TC filter add dev $UP parent 1:0 protocol ip prio 1 u32 match 
ip src ${i}/32 flowid 1:${CONT}
done
------------------------end - banda.up -------------------------
_______________________________________________
LARTC mailing list
LARTC@xxxxxxxxxxxxxxx
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Configuring a QoS Box + Cliente Bandwidth Control

Linux Advanced Routing and Traffic Control