Search squid archive

Re: Huge memory required for squid 3.5

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,


Its not disk cache, its due to in memory SSL context.


Nil


From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> on behalf of Yuri <yvoinov@xxxxxxxxx>
Sent: Wednesday, May 3, 2017 11:55 AM
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Huge memory required for squid 3.5
 

How big disk cache(s) and how it full?


03.05.2017 17:54, Nil Nik пишет:
Hi,


NO_DEFAULT_CA doesn't help. Still goes in GB. Can anyone tell me area so that i can work on?


Regards,

Nil



From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> on behalf of Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
Sent: Wednesday, April 26, 2017 7:37 PM
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Huge memory required for squid 3.5
 
On 04/26/2017 09:35 AM, Yuri Voinov wrote:

> This is openssl issue or squid's?

AFAIK, the underlying issue (i.e., bug #4005) is mostly a Squid problem:
Squid is caching SSL contexts (instead of certificates) and does a poor
job maintaining that cache.

Earlier OpenSSL versions (that had to be used when the original code was
written) complicated solving this problem. OpenSSL v1.0.1+ added APIs
that simplify some aspects of the anticipated fix. Certain OpenSSL
aspects will continue to hurt Squid, even with OpenSSL v1.0.1, but if
you want to blame a single project (instead of both), blame Squid.


> Why sessions can't share CA's data cached in memory? shared_ptr invented
> already.

OpenSSL knew how to share things well before std::shared_ptr became
available. However, it is the responsibility of the application to tell
OpenSSL what to create from scratch and what to share. A part of the
problem is that Squid tells OpenSSL to create many large things from
scratch and then caches those large things while underestimating their
size by several(?) orders of magnitude (and probably also missing many
cache hits).

More details, including the difference between problems associated with
from-client and to-server connections, are documented in the "Memory
Usage" section of http://wiki.squid-cache.org/Features/SslBump
wiki.squid-cache.org
Squid-in-the-middle decryption and encryption of straight CONNECT and transparently redirected SSL traffic, using configurable CA certificates.


FWIW, we have spent a lot of resources on triaging this problem and
drafting possible solutions (in various overlapping areas), but there is
currently no sponsor to finalize and implement any of the fixes. AFAIK,
bug #4005 is stuck.

I am glad that NO_DEFAULT_CA helps mitigate some of the problems in some
environments.


HTH,

Alex.


> 26.04.2017 9:08, Amos Jeffries пишет:
>> On 26/04/17 10:53, Yuri Voinov wrote:
>>> Ok, but how NO_DEFAULT_CA should help with this?
>>
>> It prevents OpenSSL copying that 1MB into each incoming client
>> connections memory. The CAs are only useful there when you have some
>> of the global CAs as root for client certificates - in which case you
>> still only want to trust the roots you paid for service and not all of
>> them.
>>
>> Just something to try if there are huge memory issues with TLS/SSL
>> proxying. The default behaviour is fixed for Squid-4 with the config
>> options changes. But due to being a major surprise for anyone already
>> relying on global roots for client certs it remains a problem in 3.5.
>>
>> Amos
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users@xxxxxxxxxxxxxxxxxxxxx
>> http://lists.squid-cache.org/listinfo/squid-users
lists.squid-cache.org
squid-users -- General discussion relating to Squid. The membership of this list is thousands of Squid users from around the world About squid-users

>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users@xxxxxxxxxxxxxxxxxxxxx
> http://lists.squid-cache.org/listinfo/squid-users
lists.squid-cache.org
squid-users -- General discussion relating to Squid. The membership of this list is thousands of Squid users from around the world About squid-users

>

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
lists.squid-cache.org
squid-users -- General discussion relating to Squid. The membership of this list is thousands of Squid users from around the world About squid-users



_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux