Thank you Darryl,One thing I didn't ask before and had presumed from the configuration, does your VoIP data get sent over the VPN ? I had presumed yes.
Hopefully the below will help clarify.
(below was run on the site1 firewall) Does the RX line look bad to you for Eth0?
ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:D0:CF:01:A6:52 inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:20061386 errors:206047 dropped:5857547
overruns:138637 frame:0
TX packets:8390751 errors:0 dropped:0 overruns:1 carrier:0
collisions:0 txqueuelen:100 Interrupt:5
eth1 Link encap:Ethernet HWaddr 00:D0:CF:01:A6:53 inet addr:24.239.167.x Bcast:24.237.169.255
Mask:255.255.254.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4246757 errors:0 dropped:0 overruns:0 frame:0
TX packets:7470351 errors:0 dropped:0 overruns:6 carrier:0
collisions:48974 txqueuelen:100 Interrupt:8
ipsec0 Link encap:Ethernet HWaddr 00:D0:CF:01:A6:53 inet addr:24.239.167.x Mask:255.255.254.0
UP RUNNING NOARP MTU:16260 Metric:1
RX packets:625899 errors:0 dropped:1 overruns:0 frame:0
TX packets:658838 errors:373 dropped:1 overruns:0 carrier:373
collisions:0 txqueuelen:10
(external IP has been modified)
"640kbps" is the quoted sDSL speed from the ISP, actually see about 580kbps. The DSL connections are attached to the etho on each firewall,
I would guess from your configuration of site1 the Internet is presented to you by ethernet on eth1 and that its the only thing on that interface ?
Your output of eth0 maybe because you are running a mismatching ifconfig tools version to the kernel version, check the response from 'cat /proc/net/dev' manually.
IPSEC tunnel configured as a tunnel to the "remote network" and not as aI'm not familar with SnapGear, I would like to know what VoIP handsets you are running ?
gateway (this is done on both sides -site1 & site2). Site1 VPN endpoint
is a SnapGear SME550, site 2 is a SnapGear Pro. Both firewall have a
640kbps SDSL connection.
I'm not sure what you mean by "remote network" as opposed to "gateway". Since the definition of a "gateway" means to provide access to a "remote network".
If you have any ideas on what rules I should try please let me know!I don't know if this will even run, what I'm trying to say would look something like this:
####################################################################### # # Remote site end (with 3 phones) # # Mark VoIP protocol for special treatment iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.1.20 --set-tos 0x10 iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.1.21 --set-tos 0x10 iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.1.22 --set-tos 0x10
# eth1, which is your Internet/DSL interface
# Setup the basic reservation of 240kbit
tc qdisc add dev eth1 root handle 1: htb default 20 tc class add dev eth1 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k tc class add dev eth1 parent 1:1 classid 1:10 htb rate 240kbit tc class add dev eth1 parent 1:2 classid 1:20 htb rate 400kbit
# Match IP Protocol type 47 (Check your VPN technology uses this protocol ID
# number in the IP header)
# Match Low Through put TOS (Check the inner IP-TOS value is copied into the
# outer IP-TOS value by your protocol stack, use # "tcpdump -i eth1 -n -s 1500 -v" and capture some VPN data with UDP inside)
# Match the IP address of the VPN tunnel endpoint, however if you are on DSL
# with dynamic IP then you might as well scrap this value, otherwise change
# "0.0.0.0/0" to just "10.0.0.something/32" or remove
# "match ip src 0.0.0.0/0" completely.
# This would enforce the IP of the tunnel endpoint gets special treatment so
# that no one else on the lan (like peoples workstations) couldn't start
# using their own VPN with TOS=0x10 and take this bandwidth.
tc filter add dev eth1 parent 1: prio 1 protocol ip u32 match ip protocol 47 0xff match ip tos 0x10 0xff match ip src 0.0.0.0/0 flowid 1:10
# Everything else gets flowid 1:20 tc filter add dev eth1 parent 1: prio 2 protocol ip prio 21 u32 match ip dst 0.0.0.0/0 flowid 1:20
##################################################################### # # Corp site end # # Mark VoIP protocol from gatekeeper/switch for special treatment iptables -A PREROUTING -t mangle -j TOS -p udp -s 10.0.0.7 --set-tos 0x10
# eth1, which is your Internet/DSL interface
# Setup the basic reservation of 240kbit
tc qdisc add dev eth1 root handle 1: htb default 20 tc class add dev eth1 parent 1: classid 1:1 htb rate 600kbit ceil 640kbit burst 2k tc class add dev eth1 parent 1:1 classid 1:10 htb rate 240kbit tc class add dev eth1 parent 1:2 classid 1:20 htb rate 400kbit
# Match IP Protocol type 47 (Check your VPN technology uses this protocol ID
# number in the IP header)
# Match Low Through put TOS (Check the inner IP-TOS value is copied into the
# outer IP-TOS value by your protocol stack, use # "tcpdump -i eth1 -n -s 1500 -v" and capture some VPN data with UDP inside)
# Match the IP address of the VPN tunnel endpoint, however if you are on DSL
# with dynamic IP then you might as well scrap this value, otherwise change
# "0.0.0.0/0" to just "10.0.0.something/32" or remove
# "match ip src 0.0.0.0/0" completely.
# This would enforce the IP of the tunnel endpoint gets special treatment so
# that no one else on the lan (like peoples workstations) couldn't start
# using their own VPN with TOS=0x10 and take this bandwidth.
tc filter add dev eth1 parent 1: prio 1 protocol ip u32 match ip protocol 47 0xff match ip tos 0x10 0xff match ip src 0.0.0.0/0 flowid 1:10
# Everything else gets flowid 1:20 tc filter add dev eth1 parent 1: prio 2 protocol ip prio 21 u32 match ip dst 0.0.0.0/0 flowid 1:20
You have to subscribe and reply from the address you have subscribed as.
PS. I am new to posting, and would like to update the thread, but I
don't know how to a reply to the thread, I seem to only make a new post.
List-Subscribe: <http://mailman.ds9a.nl/mailman/listinfo/lartc>, <mailto:lartc-request@xxxxxxxxxxxxxxx?subject=subscribe>
Thanks for your suggestions, I will take a look at the Wondershaper, IGood use on low bandwidth links.
saw some articles mention it but I didn't know if I could use it on the
firewalls.
-----Original Message-----
From: Darryl Miles [mailto:lartc-list@xxxxxxxxxxxx] Sent: Thursday, December 04, 2003 10:39 AM
To: Greg Freeman
Cc: lartc@xxxxxxxxxxxxxxx
Subject: Re: $100 USD to the first person that can provide the
rules/scripts that will solve the QOS latency & bandwidth allocation
issue !!!!
Greg,
You do not say in your original article how your 640kbit/640kbit link is attached, eth0, eth1 or pppX? If the interface is dedicated only for WAN usage, or if the WAN be via a gateway? If your VPN endpoint hosts are on the same or a different box to that with the 640kbit link.
You did not say if pinging between the two tunnel endpoint addresses (i.e. their non-10.x.x.x addresses) also had any latency reduction during the times of increased latency through the tunnel. I'm trying to establish if the 640kbit link is the point where the queuing occurs, or if its never saturated and the problem with your TC rule set.
Since I expect the 640kbit link is your bottle neck, and that there is a quantity of non-VPN traffic also passing through this point (web browsing, email, internet downloading), rules MUST also be applied here.
It is this point causing your latency issue, the device interface queue
is whats holding all the packets and thus adding propagation delay. While the traffic going into the IPSEC it being controlled, there is
nothing I can see in your configuration which arbitrates bandwidth usage
between webbrowsing and VPN data over the 640kbit link.
I am not the most knowledgable person about VPN and TC operation, but I
would look for a solution using ipchains to MARK the DS/TOS field of the
outside IP header of VPN packets only for VoIP/HiPriority packets, this
has to be done on the box which is a VPN endpoint at the point each
packet enters the tunnel. Maybe the inside TOS field is already copied
into the outside TOS field during encapsulation (this is what I believe
IPIP and GRE encaps would do). Use tcpdump on both the inside and
outside interfaces, if possible to see. I would expect the UDP packets
to already be marked with a "low latency" TOS (due to the nature of the
applications), if this is the case then you have enough information to
distinguish your important traffic from the rest over the 640kbit link. Other than this suggestion I'm not sure how you can mark the outside IP
packet header based on the inside IP, UDP/TCP content during
encapsulation (I'd like to know myself). I do not think any TC rules
need to be applied to the ipsec0 interface, I believe you only need to
ensure the packets have been marked at this point.
The next thing to do is, at the box with the 640kbit link, use your TC rule set to provide bandwidth reservation based on the protocol byte (VPN traffic, ESP/AH, etc..) and the IP DS/TOS header marking to select them to be placed in a higher priority queue.
Don't forget to account with the packet length increase due to the extra layer of headers of VPN traffic, both in overall bandwidth calculation and average packet length.
If your 640kbit link were say DSL and your DSL providers equipment was between the two 640kbit link interfaces, then there are also other issues with managing the inbound packet queue length at your ISP. Check out the "WonderShaper" (on google) for information on the problem and how the TC rules in it help the situation, but basically your Internet downloads need to be controlled to prevent queueing by your ISP when it sends inbound data to you.
Hope I read your original article correctly and am I'm pitching at the real problem,
Darryl
_______________________________________________ LARTC mailing list / LARTC@xxxxxxxxxxxxxxx http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
