what about using ingress policeing instead of rshaper for limiting incomming bandwidth? Dimitar ----- Original Message ----- From: "Brian Wilson" <wilson@xxxxxxxxxxxxxxxxx> To: <lartc@xxxxxxxxxxxxxxx> Sent: вторник, 05 Декември 2000 г. 17:39 Subject: [LARTC] Limiting Outgoing and Incoming Bandwidth in Linux 2.2 > After looking high and low for information on how to do this, I > finally gathered enough information to do what I wanted to do. > Not to mention, the archives from this list helped out greatly (even > though the list is fairly new). So, I figured I'd post the results of > my endeavor for those of you who are interested. I hope it helps > anyone who is also trying to do this. I'll also be housing this > document & changes here: http://bubba.org/cbq.html > > I look forward to seeing other informative posts on this list > involving Linux advanced routing concepts and traffic control > issues and appreciate the previous posts which helped me get to > this point. > > Regards, > Brian > > -- > > Limiting Outgoing and Incoming Bandwidth in Linux (for 2.2+ Kernels) > Brian Wilson - wilson@xxxxxxxxxxxxxxxxx - 12/3/2000 > > > Overview > ----------- > The point of this document is to walk you through setting up bandwidth > limiting for outgoing and incoming network packets under Linux 2.2.x+ > kernel series. The limiting of the outgoing bandwidth is accomplished > using the built-in QoS Class Based Queuing (CBQ) functions in the 2.2.x > kernel. The limiting of the incoming bandwidth is accomplished using a > stand-alone kernel module called rshaper. NOTE: rshaper also requires > slight modififications to your network driver source code and it not > officially part of the 2.2 kernel (so use it at your own risk). > > > Before You Begin > ------------------- > You first need to download a few utilities. > - iproute2 > > ftp://ftp.funet.fi/pub/mirrors/ftp.inr.ac.ru/ip-routing/iproute2-current.tar > .gz > (and if you wish to limit incoming traffic as well, you'll need rshaper) > - rshaper > ftp://ftp.prosa.it/pub/software/rshaper-1.07.tar.gz > > > Installation > ------------ > You'll first need to configure your kernel for QoS and CBQ. This is > accomplished under the "Networking Options" menu. Here you need to enable > Routing Messages and the Kernel/User netlink socket. Next, under "QoS > and/or Fair Queuing" menu, it is probably best to enable all of the options. > Below is the Networking Options portion of my kernel config. You should > probably mimic this setup (unless you want to build them as modules). If > you do choose to build them as modules, then you'll have to load all the > specific QoS modules (sch_cbq sch_tbf sch_sfq sch_prio cls_u32) after your > machine boots (or allow them to be autoloaded). Since I don't like dealing > with a slew of modules, I usually compile everything in statically. > > <*> Packet socket > [*] Kernel/User netlink socket > [*] Routing messages > <*> Netlink device emulation > [*] Network firewalls > [*] Socket Filtering > <*> Unix domain sockets > [*] TCP/IP networking > [*] IP: multicasting > [*] IP: advanced router > [*] IP: policy routing > [*] IP: equal cost multipath > [*] IP: use TOS value as routing key > [ ] IP: verbose route monitoring > [ ] IP: large routing tables > [*] IP: fast network address translation > [ ] IP: kernel level autoconfiguration > [*] IP: firewalling > [ ] IP: firewall packet netlink device > [*] IP: use FWMARK value as routing key > [*] IP: masquerading > --- Protocol-specific masquerading support will be built as > modules. > [*] IP: ICMP masquerading > --- Protocol-specific masquerading support will be built as > modules. > [*] IP: masquerading special modules support > <*> IP: ipautofw masq support (EXPERIMENTAL) > <*> IP: ipportfw masq support (EXPERIMENTAL) > <*> IP: ip fwmark masq-forwarding support (EXPERIMENTAL) > [ ] IP: optimize as router not host > < > IP: tunneling > [*] IP: aliasing support > [ ] IP: ARP daemon support (EXPERIMENTAL) > [*] IP: TCP syncookie support (not enabled per default) > --- (it is safe to leave these untouched) > <*> IP: Reverse ARP > [*] IP: Allow large windows (not recommended if <16Mb of memory) > < > The IPv6 protocol (EXPERIMENTAL) > --- > < > The IPX protocol > < > Appletalk DDP > < > CCITT X.25 Packet Layer (EXPERIMENTAL) > > < > LAPB Data Link Driver (EXPERIMENTAL) > [ ] Bridging (EXPERIMENTAL) > [ ] 802.2 LLC (EXPERIMENTAL) > < > Acorn Econet/AUN protocols (EXPERIMENTAL) > < > WAN router > [*] CPU is too slow to handle full bandwidth > QoS and/or fair queueing ---> > [*] QoS and/or fair queueing > <*> CBQ packet scheduler > <*> CSZ packet scheduler > <*> The simplest PRIO pseudoscheduler > <*> RED queue > <*> SFQ queue > <*> TEQL queue > <*> TBF queue > [*] QoS support > [*] Rate estimator > [*] Packet classifier API > <*> Routing table based classifier > <*> Firewall based classifier > <*> U32 classifier > <*> Special RSVP classifier > < > Special RSVP classifier for IPv6 > [*] Ingres traffic policing > > While you are in the kernel config, if you have decided to use rshaper and > limit incoming traffic, then go ahead and modularize your network device > driver (if its not already a module). > > < M > Sun LANCE support > > Doing so will be helpful in modifying the driver and setting it up for use > with rshaper. If you aren't going to be using rshaper, then you may skip > the > next step. > > > Installing rshaper > ------------------ > Unpackage the source, "make" and then read the README! (from the README) > (1) in global space add the following line: > int (*net_shaper_rx_hook)(struct sk_buff *skb) = NULL; > > (2) Only for Linux-2.2 (not for Linux-2.0), add this too: > EXPORT_SYMBOL(net_shaper_rx_hook) > this line must appear after the definition of the hook. > > (3) in the receive function of the driver change > netif_rx(skb) > to > if (net_shaper_rx_hook) > (*net_shaper_rx_hook)(skb); > else > netif_rx(skb); > > This should be it unless you have more than one network device or if you are > not compiling your network driver as a module. If this is the case, please > see the README included with rshaper. > > > Enabling Changes > ------------------- > Compile your kernel and modules, install them and reboot if necessary. > > Now, we'll setup our outgoing bandwidth limitations (note, replace eth0 with > your network device). > > Setup the queue, with your max bandwidth, mine is 10Mbit > # tc qdisc add dev eth0 root handle 10: cbq bandwidth 10Mbit avpkt 1000 > > Add your class. Again, bandwidth is 10Mbit. weight should be bandwidth / > 10. > # tc class add dev eth0 parent 10:0 classid 10:1 cbq bandwidth 10Mbit \ > rate 10Mbit allot 1514 weight 1Mbit prio 8 maxburst 20 avpkt 1000 > > Add your subclass where you'll limit the bandwidth. I'm limiting at a rate > of 1024Kbit's, weight should be less than the limit, preferrably by a factor > of 10 as above. > # tc class add dev eth0 parent 10:1 classid 10:100 cbq bandwidth 10Mbit \ > rate 1024kbit allot 1514 weight 128Kbit prio 5 maxburst 20 avpkt 1000 \ > bounded > > We've added the queues, now you must tell the kernel they exist and > how to manage them. We'll be using stochastic fairness queuing (sfq) > # tc qdisc add dev eth0 parent 10:100 sfq quantum 1514b perturb 15 > > Now we have to tell the kernel to send packets to the queue. Notice > the ip/mask. Change this depending on your target network to filter. > # tc filter add dev eth0 parent 10:0 protocol ip prio 100 u32 match ip \ > dst 192.168.1.0/24 flowid 10:100 > > If you get "RTNETLINK answers: Invalid argument" errors, chances are you > didn't compile in all the necessary items into the kernel. Check my > config above and make sure necessary things are there. For instance, if > the last "tc" command above delivers this error, it means that QoS > u32 queuing is not compiled into the kernel or the module isn't loaded. > > > If you do not see any errors, then chances are you have correctly limited > the outgoing bandwidth of the above device. Issue the "tc -d qdisc" > command. > It should look similar to this: > # tc -d qdisc > qdisc sfq 8001: dev eth0 quantum 1514b limit 128p flows 128/1024 perturb > 15sec > qdisc cbq 10: dev eth0 rate 10Mbit cell 8b (bounded,isolated) prio \ > no-transmit/8 weight 10Mbit allot 1514b level 2 ewma 5 avpkt 1000b maxidle > 23us > > You can verify this by connecting to your machine from a remote machine and > comparing your download speeds. This is best done on an internal LAN where > you're speeds are pretty much guaranteed max speeds. > > If you have chosen to also limit incoming bandwidth using the rshaper > utility, > then you'll need to load the rshaper.o module at this time. > # insmod /usr/lib/rshaper.o > > Then you need to setup your incoming bandwidth limitation using the > rshaperctl > utility. For more information on this, see the rshaper README. In this > case > my local network is 192.168.1.0/24. > # rshaperctl 192.168.1.0/24 128000 > > This shapes all incoming traffic on the local network to 128kb/s. Running > rshapectl with no options will show you the status of shaped hosts. > # rshaperctl > host mask bytes-per-sec max-queue-len (s) > 192.168.1.0 255.255.255.0 128000 4 > > To test this setup, connect from your local machine to a remote machine and > compare your download speeds from this machine. Again, this is best done > on an internal LAN where you're speeds are pretty much guaranteed. > > > Script > ------ > Here is the script I use to start cbq on my machine: > ---------------- > #!/bin/sh > > FILTERMASK="192.168.1.0/24" > > # bandwidth in Kbits > IN_BANDWIDTH="1024" > > # bandwidth in bytes > OUT_BANDWIDTH="128000" > > DEV="eth0" > > PATH=/usr/sbin:/sbin > > echo -n "Shaping incoming traffic to ${IN_BANDWIDTH}Kbit/s " > tc qdisc add dev $DEV root handle 10: cbq bandwidth 10Mbit avpkt 1000 > > echo -n . > > tc class add dev $DEV parent 10:0 classid 10:1 cbq bandwidth 10Mbit rate > 10Mbit allot 1514 weight 1Mbit prio 8 maxburst 20 avpkt 1000 > echo -n . > > tc class add dev $DEV parent 10:1 classid 10:100 cbq bandwidth 10Mbit rate > ${IN_BANDWIDTH}kbit allot 1514 weight 85kbit prio 5 maxburst 20 avpkt 1000 > bounded > > echo -n . > > tc qdisc add dev $DEV parent 10:100 sfq quantum 1514b perturb 15 > > echo -n . > > tc filter add dev $DEV parent 10:0 protocol ip prio 100 u32 match ip dst > $FILTERMASK flowid 10:100 > > echo -n . > echo > > echo -n "Shaping outgoing traffic to ${OUT_BANDWIDTH}bytes/s " > modprobe rshaper > rshaperctl $FILTERMASK $OUT_BANDWIDTH > echo -n . > echo > ------------------- > > Questions/Comments/Information > ----------------------------------- > The information contained herein was gathered from a weekends worth of > searches on google. Hopefully you'll find it useful and usable. I > claim to be no expert on this topic by any means, so I don't know if > I'll be much help if you run into problems, but I'll do my best to help. > So please direct comments and questions to me, Brian Wilson > wilson@xxxxxxxxxxxxxxxxxx > > > > Credits > --------- > http://snafu.freedom.org/linux2.2/iproute-notes.html > http://www.ibiblio.org/pub/Linux/docs/HOWTO/other-formats/html_single/Adv-Ro > uting-HOWTO.html#s8 > http://mailman.ds9a.nl/pipermail/lartc/2000q4/ > z3d and sh00p for pointing fingers and laughing. > > > > > _______________________________________________ > LARTC mailing list / LARTC@xxxxxxxxxxxxxxx > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/ >
