I have setup like below :-
end user >> LinuxProxy(3.1.10) >> External Proxy(3.4)>> Internet
- when we configure external proxy ip in end user's IE, all HTTP & HTTPS site work properly.
- when we configure Linux proxy ip in end user's IE, HTTP works fine but none of the HTTPS site open.
- when we access https site from LinuxProxy, below logs are appearing in access.log
TCP_MISS/503 0 CONNECT sitename:443 username@xxxxxxxxxxxx DIRECT / - -
TCP_MISS/200 23456 GET http://www.anysite.com username@xxxxxxxxxxxx DEFAULT_PARENT/10.10.x.x text/html
what I making out from log is ( I might be wrong) - HTTPS request are going directly instead .
attached is my Linux Proxy config-
=================
#
# Recommended minimum configuration:
#
#auth_param negotiate program /usr/lib64/squid/squid_kerb_auth -d -s GSS_C_NO_NAME
auth_param negotiate program /usr/lib64/squid/squid_kerb_auth -s HTTP/proxy02.abcd.com@xxxxxxxx -d
auth_param negotiate children 10
auth_param negotiate keep_alive on
auth_param basic credentialsttl 2 hours
acl ad_auth proxy_auth REQUIRED
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
#
# Recommended minimum Access Permission configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
#
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
#http_access allow localnet
#http_access allow localhost
http_access deny !ad_auth
http_access allow ad_auth
# And finally deny all other access to this proxy
http_access deny all
# Squid normally listens to port 3128
http_port 8080
cache_peer xx.xx.2.108 parent 8080 0 default
#dns_nameservers ABCDNS.ABCD.COM
dns_nameservers xx.xx.2.108
# We recommend you to use at least the following line.
#hierarchy_stoplist cgi-bin ?
# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /var/spool/squid 2048 16 256
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid
# Log forwarding to SysLog
access_log syslog:local1.info
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
-======================================
Thanks & Regards
Nilesh Suresh Gavali
From: squid-users-request@xxxxxxxxxxxxxxxxxxxxx
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Date: 14/06/2016 13:00
Subject: squid-users Digest, Vol 22, Issue 62
Sent by: "squid-users" <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx>
Send squid-users mailing list submissions to
squid-users@xxxxxxxxxxxxxxxxxxxxx
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.squid-cache.org/listinfo/squid-users
or, via email, send a message with subject or body 'help' to
squid-users-request@xxxxxxxxxxxxxxxxxxxxx
You can reach the person managing the list at
squid-users-owner@xxxxxxxxxxxxxxxxxxxxx
When replying, please edit your Subject line so it is more specific
than "Re: Contents of squid-users digest..."
Today's Topics:
1. Re: Excessive TCP memory usage (Eliezer Croitoru)
2. Re: Squid not allowing HTTPS access (Nilesh Gavali)
3. Re: Squid not allowing HTTPS access (Antony Stone)
----------------------------------------------------------------------
Message: 1
Date: Tue, 14 Jun 2016 13:21:32 +0300
From: "Eliezer Croitoru" <eliezer@xxxxxxxxxxxx>
To: "'Deniz Eren'" <denizlist@xxxxxxxxxxxxx>,
<squid-users@xxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Excessive TCP memory usage
Message-ID: <05c801d1c626$85a97ff0$90fc7fd0$@ngtech.co.il>
Content-Type: text/plain; charset="utf-8"
Hey,
Steps to reproduce are not exactly everything since squid works fine in many other scenarios.
I do not know this specific system but if you are talking about 1-4k open connections it should not be a big problem for many servers.
The issue in hands is a bit different.
Have you tried tuning the ipv4\net using sysctl to see if it affects anything?
What I can offer is to build a tiny ICAP service that will use a 204 on every request and then moves on.
If the same happens with the dummy service it's probably a very bad scenario and if not then we can try to think
if there is something unique about your setup.
I have not seen this issue in my current testing setup which includes 3.5.19 + ICAP url filtering service.
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer@xxxxxxxxxxxx
-----Original Message-----
From: squid-users [mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Deniz Eren
Sent: Tuesday, June 14, 2016 11:07 AM
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Excessive TCP memory usage
Little bump :)
I have posted bug report with steps to reproduce. The problem still exists and I am curious whether anyone else is having the same problem, too.
http://bugs.squid-cache.org/show_bug.cgi?id=4526
On Wed, May 25, 2016 at 1:18 PM, Deniz Eren <denizlist@xxxxxxxxxxxxx> wrote:
> When I listen to connections between squid and icap using tcpdump I
> saw that after a while icap closes the connection but squid does not
> close, so connection stays in CLOSE_WAIT state:
>
> [root@test ~]# tcpdump -i any -n port 34693
> tcpdump: WARNING: Promiscuous mode not supported on the "any" device
> tcpdump: verbose output suppressed, use -v or -vv for full protocol
> decode listening on any, link-type LINUX_SLL (Linux cooked), capture
> size 96 bytes
> 13:07:31.802238 IP 127.0.0.1.icap > 127.0.0.1.34693: F
> 2207817997:2207817997(0) ack 710772005 win 395 <nop,nop,timestamp
> 104616992 104016968>
> 13:07:31.842186 IP 127.0.0.1.34693 > 127.0.0.1.icap: . ack 1 win 3186
> <nop,nop,timestamp 104617032 104616992>
>
> [root@test ~]# netstat -tulnap|grep 34693
> tcp 215688 0 127.0.0.1:34693 127.0.0.1:1344
> CLOSE_WAIT 19740/(squid-1)
>
> These CLOSE_WAIT connections do not timeout and stay until squid
> process is killed.
>
> 2016-05-25 10:37 GMT+03:00 Deniz Eren <denizlist@xxxxxxxxxxxxx>:
>> 2016-05-24 21:47 GMT+03:00 Amos Jeffries <squid3@xxxxxxxxxxxxx>:
>>> On 25/05/2016 5:50 a.m., Deniz Eren wrote:
>>>> Hi,
>>>>
>>>> After upgrading to squid 3.5.16 I realized that squid started using
>>>> much of kernel's TCP memory.
>>>
>>> Upgrade from which version?
>>>
>> Upgrading from squid 3.1.14. I started using c-icap and ssl-bump.
>>
>>>>
>>>> When squid was running for a long time TCP memory usage is like below:
>>>> test@test:~$ cat /proc/net/sockstat
>>>> sockets: used *
>>>> TCP: inuse * orphan * tw * alloc * mem 200000
>>>> UDP: inuse * mem *
>>>> UDPLITE: inuse *
>>>> RAW: inuse *
>>>> FRAG: inuse * memory *
>>>>
>>>> When I restart squid the memory usage drops dramatically:
>>>
>>> Of course it does. By restarting you just erased all of the
>>> operational state for an unknown but large number of active network connections.
>>>
>> That's true but what I mean was squid's CLOSE_WAIT connections are
>> using too much memory and they are not timing out.
>>
>>> Whether many of those should have been still active or not is a
>>> different question. the answer to which depends on how you have your
>>> Squid configured, and what the traffic through it has been doing.
>>>
>>>
>>>> test@test:~$ cat /proc/net/sockstat
>>>> sockets: used *
>>>> TCP: inuse * orphan * tw * alloc * mem 10
>>>> UDP: inuse * mem *
>>>> UDPLITE: inuse *
>>>> RAW: inuse *
>>>> FRAG: inuse * memory *
>>>>
>>>
>>> The numbers you replaced with "*" are rather important for context.
>>>
>>>
>> Today again I saw the problem:
>>
>> test@test:~$ cat /proc/net/sockstat
>> sockets: used 1304
>> TCP: inuse 876 orphan 81 tw 17 alloc 906 mem 29726
>> UDP: inuse 17 mem 8
>> UDPLITE: inuse 0
>> RAW: inuse 1
>> FRAG: inuse 0 memory 0
>>
>>>> I'm using Squid 3.5.16.
>>>>
>>>
>>> Please upgrade to 3.5.19. Some important issues have been resolved.
>>> Some of them may be related to your TCP memory problem.
>>>
>>>
>> I have upgraded now and problem still exists.
>>
>>>> When I look with "netstat" and "ss" I see lots of CLOSE_WAIT
>>>> connections from squid to ICAP or from squid to upstream server.
>>>>
>>>> Do you have any idea about this problem?
>>>
>>> Memory use by the TCP system of your kernel has very little to do
>>> with Squid. Number of sockets in CLOSE_WAIT does have some relation
>>> to Squid or at least to how the traffic going through it is handled.
>>>
>>> If you have disabled persistent connections in squid.conf then lots
>>> of closed sockets and FD are to be expected.
>>>
>>> If you have persistent connections enabled, then fewer closures
>>> should happen. But some will so expectations depends on how high the
>>> traffic load is.
>>>
>> Persistent connection parameters are enabled in my conf, the problem
>> occurs especially with connections to c-icap service.
>>
>> My netstat output is like this:
>> netstat -tulnap|grep squid|grep CLOSE
>>
>> tcp 211742 0 127.0.0.1:55751 127.0.0.1:1344
>> CLOSE_WAIT 17076/(squid-1)
>> tcp 215700 0 127.0.0.1:55679 127.0.0.1:1344
>> CLOSE_WAIT 17076/(squid-1)
>> tcp 215704 0 127.0.0.1:55683 127.0.0.1:1344
>> CLOSE_WAIT 17076/(squid-1)
>> ...(hundreds)
>> Above ones are connections to c-icap service.
>>
>> netstat -tulnap|grep squid|grep CLOSE Active Internet connections
>> (servers and established)
>> Proto Recv-Q Send-Q Local Address Foreign Address
>> State PID/Program name
>> tcp 1 0 192.168.2.1:8443 192.168.6.180:45182
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.2.177:50020
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.2.172:60028
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.6.180:44049
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.6.180:55054
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.2.137:52177
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.6.180:43542
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.6.155:39489
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.0.147:38939
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.6.180:38754
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.0.164:39602
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.0.147:54114
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.6.180:57857
>> CLOSE_WAIT 15245/(squid-1)
>> tcp 1 0 192.168.2.1:8443 192.168.0.156:43482
>> CLOSE_WAIT 15245/(squid-1)
>> ...(about 50)
>> Above ones are connections from https port to client.
>>
>> As you can see recv-q for icap connections allocate more memory but
>> connections from https_port to upstream server connections allocate
>> only one byte.
>>
>> What can be done to close these unused connections?
>>
>> The problem in this thread seems similar:
>> http://www.squid-cache.org/mail-archive/squid-users/201301/0092.html
>>
>>> Amos
>>>
>>> _______________________________________________
>>> squid-users mailing list
>>> squid-users@xxxxxxxxxxxxxxxxxxxxx
>>> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
------------------------------
Message: 2
Date: Tue, 14 Jun 2016 11:36:30 +0100
From: Nilesh Gavali <nilesh.gavali@xxxxxxx>
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Squid not allowing HTTPS access
Message-ID:
<OF83BE7BF7.92EBA289-ON80257FD2.003A3940-80257FD2.003A455E@xxxxxxx>
Content-Type: text/plain; charset="utf-8"
Team;
kindly help on below issue.
Thanks & Regards
Nilesh Suresh Gavali
From: Nilesh Gavali/MUM/TCS
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Date: 13/06/2016 14:00
Subject: Squid not allowing HTTPS access
Hello All;
Facing issue while accessing HTTPS via squid, normal http traffic working
fine. I have squid 3.1.10 on RHEL.6.0
attached is my squid .conf for your reference,..help will be much
appreciated.
Thanks & Regards
Nilesh Suresh Gavali
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160614/79185c58/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: squid.conf
Type: application/octet-stream
Size: 3149 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160614/79185c58/attachment-0001.obj>
------------------------------
Message: 3
Date: Tue, 14 Jun 2016 12:40:14 +0200
From: Antony Stone <Antony.Stone@xxxxxxxxxxxxxxxxxxxx>
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Squid not allowing HTTPS access
Message-ID: <201606141240.15034.Antony.Stone@xxxxxxxxxxxxxxxxxxxx>
Content-Type: Text/Plain; charset="iso-8859-15"
On Tuesday 14 June 2016 at 12:36:30, Nilesh Gavali wrote:
> Team;
> kindly help on below issue.
Nilesh:
kindly respond to my reply below.
On Monday 13 June 2016 at 15:26:22, Antony Stone wrote:
> On Monday 13 June 2016 at 15:01:02, Nilesh Gavali wrote:
> > Facing issue while accessing HTTPS via squid, normal http traffic working
> > fine.
>
> Please define "issue", with as much detail as possible:
>
> - what exactly are you trying to do when a problem occurs?
>
> - have you previously been able to do this without the problem occurring?
> If so, what has changed between then and now (different squid config,
> different squid version, different browser...)?
>
> - what is the actual problem (what error message is displayed, if there is
> one)?
>
> - what appears in your access.log when the problem occurs?
>
> - any other information you think might be relevant to us in working out
> what's happening on your network?
>
> > I have squid 3.1.10 on RHEL.6.0
>
> For HTTPS traffic in particular you are strongly advised to upgrade.
>
>
> Regards,
>
> Antony.
--
All generalisations are inaccurate.
Please reply to the list;
please *don't* CC me.
------------------------------
Subject: Digest Footer
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
------------------------------
End of squid-users Digest, Vol 22, Issue 62
*******************************************
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
