Jason wrote:
Jason wrote:
Amos Jeffries wrote:
Jason wrote:
Amos Jeffries wrote:
On Wed, 19 Aug 2009 20:58:19 -0700, Jason <jason@xxxxxxxx> wrote:
Everyone,
I am running squid 3.1.0.6, transparent/intercepting (non tproxy,
non wpad, etc), nat'ted network, and users are reporting problems
using the uploaders at the facebook website. When I explored
this, here is what I found:
Facebook has two upload methods, a newer java based one, and an
older html forms (i think) based one.
1. Both uploaders work perfectly when I bypass squid.
2. With internet explorer, the old uploader works fine
3. With Internet Explorer, the new uploader fails at first. If
you immediately hit the "Upload" button after the failure, it works.
4. With Firefox, the old uploader gives this error from squid:
ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the
URL: http://upload.facebook.com
/photos_upload.php
Connection to upload.facebook.com failed.
The system returned: (110) Connection timed out
The remote host or network may be down. Please try the request again.
Your cache administrator is yours truly.
5. With Firefox, the new uploader fails at first. If you
immediately hit the "Upload" button after a failure, it works
(just like the IE
case).
On the proxy machine:
tcp_window_scaling is off
tcp_ecn is off
.facebook.com is in the "always direct" list I maintain.
Any help in solving this would be great!
Jason
Below is my Config:
qos_flows local-hit=0x30
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl Safe_ports port 80 # http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
acl our_networks src 10.0.0.0/16
http_access allow our_networks
http_access allow localhost
acl directlist dstdomain "/etc/squid/directsites"
always_direct allow directlist
http_access deny all
http_reply_access allow our_networks
http_reply_access allow localhost
http_reply_access deny all
icp_access deny all
htcp_access deny all
htcp_clr_access deny all
miss_access allow our_networks
miss_access allow localhost
miss_access deny all
http_port 10.0.0.1:3594 transparent
disable-pmtu-discovery=transparent
http_port 127.0.0.1:3594 transparent
disable-pmtu-discovery=transparent
cache_mem 128 MB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir aufs /squida 21760 16 256
cache_dir aufs /squidb 21760 16 256
cache_dir aufs /squidc 21760 16 256
max_open_disk_fds 0
minimum_object_size 0 KB
maximum_object_size 10 MB
cache_swap_low 95
cache_swap_high 97
access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
mime_table /etc/squid/mime.conf
pid_filename /var/run/squid.pid
log_fqdn off
strip_query_terms off
unlinkd_program /usr/lib/squid/unlinkd
url_rewrite_program /usr/bin/squidGuard
url_rewrite_children 32
url_rewrite_concurrency 0
url_rewrite_host_header on
url_rewrite_bypass off
refresh_pattern (cgi-bin|\?) 0 0% 0
refresh_pattern . 0 20% 4320
quick_abort_min 50 KB
quick_abort_max 50 KB
quick_abort_pct 50
read_ahead_gap 16 KB
negative_ttl 0 minutes
positive_dns_ttl 5 minutes
negative_dns_ttl 10 seconds
range_offset_limit 0 KB
request_header_max_size 128 KB
reply_header_max_size 128 KB
ie_refresh on
request_entities on
forward_timeout 1 minutes
connect_timeout 20 seconds
shutdown_lifetime 3 seconds default
cache_mgr support@xxxxxxxx
cache_effective_user proxy
cache_effective_group proxy
visible_hostname integrityinternet.net
snmp_port 45656
snmp_access allow our_networks
snmp_access allow localhost
snmp_access deny all
snmp_incoming_address 10.0.0.1
icon_directory /usr/share/squid/icons
dns_nameservers 127.0.0.1
ipcache_size 5120
ipcache_low 95
ipcache_high 97
fqdncache_size 5120
memory_pools_limit 512 MB
client_db off
uri_whitespace strip
coredump_dir /squida
pipeline_prefetch off
client_persistent_connections off
server_persistent_connections off
Please note that "always_direct" does not mean the URLs bypass
Squid. It
means that squid will not pass those requests to a cache_peer
server. Of
which you have none, meaning the always_direct is merely wasting
CPU time.
Please try these:
* a current release of 3.1
* turning persistent connections ON.
client_persistent_connections off
server_persistent_connections off
Amos
Amos,
Thank you for replying. I've tried persistent_connections both
ways, with no difference. Next, I'll try the latest squid 3.1. Also
thanks for the tip regarding always_direct. I thought it meant that
squid would not look in its cache for that site.
Jason
Ah, to not use the local storage its the "cache" directive with ACLs
describing what not to store.
http://www.squid-cache.org/Doc/config/cache/
Amos
I've compiled and am running on squid 3.1.0.13, still having the
problems with facebook. The 3.1.0.13 error pages in squid are nicer
to look at than the 3.1.0.6 pages were, however!
Anyone have any other ideas? Could it be that something in the http
communication is breaking down?
Jason
More information: When I manually inform the web browsers of the proxy,
the facebook error is still there. So I don't think it has anything to
do with being transparent/intercepting.
Jason
The problem is definitely this:
"The system returned: (110) Connection timed out "
when running the first request the TCP link takes too long to establish.
On following requests it seems to have underlying system routing states
already found (ie DNS caches in the resolver or Squid or persistent TCP
connections already opened or firewall state about MTU and routing tabel
already stored)
It might be as simple as these being too short:
forward_timeout 1 minutes - total time allowed to search for a
working destination (opening, writing and receiving reply back from
multiple TCP links).
connect_timeout 20 seconds - maximum time allowed to open a TCP link.
Yes the ideal for web stuff is 10 sec or much less for a full forwarding
and reply. Sometimes it can take a while though when sending data.
upload.facebook.com seems to be having DNS trouble from what I can tell
right now.
host upload.facebook.com
-->
upload.facebook.com has address 69.63.178.32
;; connection timed out; no servers could be reached
Host upload.facebook.com not found: 3(NXDOMAIN)
Takes a total of 12 seconds to reply with two contradictory answers.
Your DNS settings in squid.conf may also be compounding the problem.
You are enforcing a minimum storage time for positive DNS results inside
Squid of 5 minutes (3600 seconds). Negative results ar only stored for a
minimumof 10 seconds. The facebook IPs change every 30 to 900 seconds
on some load balancing.
Playing with DNS TTLs on domains you don't own will I expect cause the
following behaviour for you:
Some DNS lookup for facebook:
-> some DNS answer comes back with an IP
--> connection made and client request happens nicely...
-> 30 seconds or so later the IP changes.
Some other request for facebook (less than 5 minutes from first)..
-> Squid checks for IPs. Forced to use the cached one.
~~> it _may_ or may not still actually work.
-> if it has not you are lucky and things work.
-> on failure Squid sends out a "connection failed" error to the
client due to bad IPs.
-> Squid marks the one IP as bad and needing replacement.
Next request for facebook arrives...
-> Squid sees its IPs are all bad. Checks negative TTL.
--> sees negative TTL as passed and new lookup needed.
-> repeat from step #1.
This is not limited to facebook, but any domain with TTL shorter than
your cache minimums. Luckily DNS best-practice is to set TTL of 24 hours
or so. Most domains will be doing that and not cause you problems.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
Current Beta Squid 3.1.0.13
