I believe it may be related to squid getting rescheduled to a different CPU and the new cpu/core might have an empty network queue (linux starts to have a separate network queue per CPU/core for newer network cards to avoid locking, which may introduce this reordering), where once core is faster at pushing out the bytes to the network card. I remember that 2 years ago we have seen something like this and the solution then was to reduce the number of cores for our virtual machine to a single CPU.. But I suspect that setting CPU affinity for the squid process via: "cpu_affinity_map process_numbers=1 cores=1" Also solves the issue. Right now we run this config on a multi-core machine and we do not see this issue. Ciao, Martin P.s: if you run a virtual machine, then this "feature" may also get introduced at the virtual network layer as well - but at least it seems to works for us with VMWare. -----Original Message----- From: Amos Jeffries [mailto:squid3@xxxxxxxxxxxxx] Sent: Sonntag, 15. Dezember 2013 04:33 To: squid-users@xxxxxxxxxxxxxxx Subject: Re: Out of sequence packets? On 15/12/2013 4:58 a.m., Matthew Goff wrote: > Hi Amos, > > First, sorry for the double post -- my email seemed to be having > issues yesterday. > > As to my issue: What steps can I do to try and validate that this is > Squid or not? When I remove the following iptables entry and bypass > Squid I can capture tcpdump traffic on the proxy machine and see no > TCP reassemblies. Leaving the rules in place and passing traffic > through Squid begins to show TCP reassemblies again and my application > no longer works. > > -A PREROUTING -p tcp -m tcp --dport 80 -j TPROXY --on-port 3128 > --on-ip 0.0.0.0 --tproxy-mark 0x1/0x1 > -A DIVERT -j MARK --set-xmark 0x1/0xffffffff > The order of those rules is extremely sensitive. The DIVERT (which handles both from-Squid and from-server packets) is required before the TPROXY (which catches packets into Squid). > I've been using my setup for a few years without issue and have never > had an application fail to work prior to this. However when the > application fails when routing traffic through Squid yet works when I > bypass Squid, I'm not sure what else to blame or where else to look. What do you mean by re-assemblies exactly... * fragmented packets being assembled is required when there is a service reading those packets as I/O. Optional for a router simply passing them on? * packets ACK not being received from server and re-sent by Squid machine TCP stack? * packets being received from client multiple times? Amos This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp
