I have several kids and, to keep them happy and myself busy, I run 6 Windows
boxes behind a Linux box that firewalls and SNAT's. Our internet connection
is ADSL with nominal line speeds of 128 kbit up and 1400 kbit down.
Everything has worked well except when one person is playing a first person
shooter like Q3, CS, RTCW, etc. and someone else is downloading, uploading,
sending mail with large attachments, etc. Whenever this occurs, shouting
and disharmony usually results.
To fix this, I recently dug through the Linux Advanced Routing & Traffic
Control HOWTO, along with several other references, and I have been able to
address all of the issues: Typical game pings now do not change at all
during heavy downloads, and increase only slightly (~10 - 20 msec) during
heavy uploads. The cost of this is about an 15% decrease in download
bandwidth, and about a 30% decrease in upload bandwidth, which I have found
to be more than acceptable.
I started with the HTB version of the "Ultimate Traffic Conditioner" script
in the HOWTO and then tweaked it a bit to reduce game traffic latency even
further, including the addition of a trivial bit of policy routing to send
all non-game packets through a default route with a very small per-route
MTU.
I have attached a script of my implementation below. I've done this a) for
the benefit of anyone searching the mailing list archive, and b) on the
chance that someone might know of a way to improve upon my results! :)
Questions, comments, and suggestions are all welcome.
My thanks to the authors of this excellent HOWTO!
Bill Powers
wepprop@sbcglobal.net
#!/bin/sh
#
# GameScript This script establishes policy routing and traffic
# control rules to minimize latency for game packets
# in the presence of other traffic.
#
# Besides this script, there is one other thing that must be done.
# Assuming that iproute2 is already installed, edit the file
# /etc/iproute2/rt_tables and add the following line at the bottom:
# "100 Small_MTU"
# ***********************************************************************
# DEFINES *
# ***********************************************************************
# Change these values as required to reflect your setup
# Addresses and Interfaces
LAN_IP_RANGE="192.168.1.0/24"
LAN_IP="192.168.0.1"
LAN_INTERFACE="eth0"
LOCALHOST_IP="127.0.0.1/32"
INTERNET_IP_RANGE="123.123.123.0/24"
INTERNET_IP="123.123.123.123"
INTERNET_GATEWAY="123.123.123.1"
INTERNET_INTERFACE="eth1"
# Executables
IPTABLES="/sbin/iptables"
TC="/sbin/tc"
IP="/sbin/ip"
# Information used to identify game traffic.
# add more as required
HOST1="192.168.0.2"
HOST1_GAME_PORT="27661"
# Packet marks (arbitrary)
GAME_PACKET="1"
# For traffic shaping:
#
# The numbers below were arrived at by test on a DSL
# line with nominal line speeds of 128 kbit up and
# 1400 kbit down. Actual measured throughput was
# about 90 kbit up and 1150 kbit down.
#
# A note regarding MTU: Standard ethernet MTU is 1500
# bytes, which which resulted in unacceptable single
# packet xmit waits of 1500 x 8 / 90,000 = 133 msec.
# Lowering the interface MTU changes the MTU in both
# directions, which helped uplink latency but hurt
# downlink throughput. Lowering the interface MTU to
# 256 bytes resulted in a downlink throughput of less
# than 500kbit. An interface MTU in the 400 - 500 byte
# range provided an acceptable compromise, with single
# packet xmit times of about 40 msec and downlink speeds
# of about 700kbit. However, leaving the interface MTU
# at 1500 bytes and setting a lower per-route MTU that
# only affected non-game uplink traffic was the best
# solution. An uplink MTU smaller than 256 bytes would
# help latency even more, but tc and/or htb don't seem
# to like mtu's below 256 and, besides, 256 results in a max
# single packet xmit wait of around 25 msec, with
# even better average behavior.
STD_MTU="1500"
TC_MTU="256"
TC_MSS=$(( $STD_MTU - 40 ))
TC_UPLINK_RATE="90"
TC_DOWNLINK_RATE="1000"
TC_GAME_RATE="30"
TC_GAME_CEIL=$TC_UPLINK_RATE
TC_OTHER_RATE=$(( $TC_UPLINK_RATE - $TC_GAME_RATE ))
TC_OTHER_CEIL=$(( $TC_UPLINK_RATE - $TC_GAME_RATE ))
# *********************************************************************
# RULES *
# *********************************************************************
case "$1" in
start)
# ***************************************************************
# MANGLE Table PREROUTING Chain *
# ***************************************************************
# Firewall packet marking TCP game traffic from Host1
$IPTABLES --table mangle \
--append PREROUTING \
--protocol TCP \
--in-interface $LAN_INTERFACE \
--source $HOST1 \
--source-port $HOST1_GAME_PORT \
--jump MARK \
--set-mark $GAME_PACKET
# Firewall packet marking UDP game traffic from Host1
$IPTABLES --table mangle \
--append PREROUTING \
--protocol UDP \
--in-interface $LAN_INTERFACE \
--source $HOST1 \
--source-port $HOST1_GAME_PORT \
--jump MARK \
--set-mark $GAME_PACKET
# Firewall packet marking TCP game traffic to Host1
$IPTABLES --table mangle \
--append PREROUTING \
--protocol TCP \
--in-interface $INTERNET_INTERFACE \
--destination $HOST1 \
--destination-port $HOST1_GAME_PORT \
--jump MARK \
--set-mark $GAME_PACKET
# Firewall packet marking UDP game traffic to Host1
$IPTABLES --table mangle \
--append PREROUTING \
--protocol UDP \
--in-interface $INTERNET_INTERFACE \
--destination $HOST1 \
--destination-port $HOST1_GAME_PORT \
--jump MARK \
--set-mark $GAME_PACKET
# ***************************************************************
# Policy Routing *
# ***************************************************************
# Delete any existing / old rules.
$IP rule del priority 4000 2> /dev/null
$IP rule del priority 5000 2> /dev/null
# Flush the alternate routing table and routing cache
$IP route flush table Small_MTU 2> /dev/null
$IP route flush cache
# Duplicate the normal routing table except lower the MTU of the
# default route.
$IP route add $LOCALHOST_IP dev lo table Small_MTU
$IP route add $LAN_IP_RANGE dev $LAN_INTERFACE src $LAN_IP \
table Small_MTU proto static
$IP route add $INTERNET_IP_RANGE dev $INTERNET_INTERFACE \
src $INTERNET_IP table Small_MTU proto static
$IP route add default via $INTERNET_GATEWAY mtu $TC_MTU \
advmss $TC_MSS table Small_MTU proto static
# Game traffic continues to go to the main routing table with
# so that it can take advantage of larger uplink packet sizes.
$IP rule add fwmark $GAME_PACKET priority 4000 table main
# Now start referring non-game traffic to the new routing table
$IP rule add from 0/0 priority 5000 table Small_MTU
$IP route flush cache
# ***************************************************************
# Uplink Traffic Control *
# ***************************************************************
# Egress bandwidth shaping and scheduling are performed to ensure
# that packets are never queued in the ADSL modem, and that game
# packets, if present, take priority over all other traffic.
# First delete any previous traffic control rules
$TC qdisc del dev $INET_IFACE root 2> /dev/null
$TC qdisc del dev $INET_IFACE ingress 2> /dev/null
# Now establish the HTB root discipline
$TC qdisc add dev $INTERNET_INTERFACE root handle 1:0 \
htb default 11 r2q 1
# Now establish the root class
$TC class add dev $INTERNET_INTERFACE parent 1:0 classid 1:1 \
htb rate $TC_UPLINK_RATE"kbit" ceil $TC_UPLINK_RATE"kbit" \
burst 6k cburst 6k
# Add leaf class for game traffic
$TC class add dev $INTERNET_INTERFACE parent 1:1 classid 1:10 \
htb rate $TC_GAME_RATE"kbit" ceil $TC_GAME_CEIL"kbit" \
prio 1 burst 6k cburst 6k
# Add leaf class for non-game traffic. Note that non-game
# traffic is capped at about 67% of the available uplink
# bandwidth, both for rate and ceiling. This was done
# to ensure that sufficient bandwidth (tokens) is always
# available for game packets when they arrive.
$TC class add dev $INTERNET_INTERFACE parent 1:1 classid 1:11 \
htb rate $TC_OTHER_RATE"kbit" ceil $TC_OTHER_CEIL"kbit" \
prio 2 mtu $TC_MTU
# Add fifo queueing discipline for game traffic
$TC qdisc add dev $INTERNET_INTERFACE parent 1:10 handle 10: \
pfifo limit 25
# Add prio queueing discipline for non-game traffic to provide
# standard TOS priority queueing.
$TC qdisc add dev $INTERNET_INTERFACE parent 1:11 handle 11: \
prio
# Add sfq queueing discipline for minimize-delay traffic
$TC qdisc add dev $INTERNET_INTERFACE parent 11:1 handle 111: \
sfq perturb 5
# Add sfq queueing discipline for best-effort traffic
$TC qdisc add dev $INTERNET_INTERFACE parent 11:2 handle 112: \
sfq perturb 5
# Add sfq queueing discipline for maximize-throughput traffic
$TC qdisc add dev $INTERNET_INTERFACE parent 11:3 handle 113: \
sfq perturb 5
# Now filter game traffic to leaf 1:10 as first priority
$TC filter add dev $INTERNET_INTERFACE parent 1:0 \
protocol ip prio 1 handle $GAME_PACKET fw flowid 1:10
# Empty ack packets are assigned directly to the minimize-
# delay queue.
$TC filter add dev $INTERNET_INTERFACE parent 11:0 protocol ip \
prio 3 u32 match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 11:1
# The remaining traffic defaults to htb leaf 1:11
# **************************************************************
# Downlink Traffic Control (Ingress Policing) *
# **************************************************************
# Downlink traffic is limited to about 85% of actual downlink
# capability to prevent upstream queueing.
# First establish an ingress qdisc
$TC qdisc add dev $INTERNET_INTERFACE handle ffff: ingress
# Incoming game traffic is not policed
$TC filter add dev $INTERNET_INTERFACE parent ffff: \
protocol ip prio 1 handle $GAME_PACKET fw flowid :1
# Filter everything else to that qdisc and drop packets
# that exceed the bandwidth limit
$TC filter add dev $INTERNET_INTERFACE parent ffff: \
protocol ip prio 3 u32 match ip src 0.0.0.0/0 \
police rate $TC_DOWNLINK_RATE"kbit" burst 3k drop \
flowid :1
;;
stop)
# Remove any uplink throttling
$TC qdisc del dev $INTERNET_INTERFACE root 2> /dev/null
$TC qdisc del dev $INTERNET_INTERFACE ingress 2> /dev/null
# Remove policy routing
$IP rule del priority 5000 2> /dev/null
$IP rule del priority 4000 2> /dev/null
$IP route flush table Small_MTU 2> /dev/null
$IP route flush cache
;;
restart)
$0 stop
sleep 3
$0 start
;;
*)
echo "Usage: ./$0 start|stop|restart}"
exit 1
esac
exit 0
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
