Search squid archive

Re: SSL intercept in explicit mode

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



AFAIK,

SSL bump subsystem uses OpenSSL memory routines. So, first of all, most probably leaks (if any) can be OpenSSL-related, but not squid itself.

Now let's see your config snippets.

13.03.2018 23:00, Aaron Turner пишет:
"Usually misconfiguration leads to memory overhead."

This may be true, but if you look in the list archives a few months
ago I basically chased my tail in circles and nobody could tell me
what I was doing wrong and so many of the docs are so old that they're
worse then useless, they seem to suggest the wrong thing.

It was literally leaking GB's worth of RAM.  I even disabled all
caching and process sizes were growing into the GB's.  Turn off
ssl-bump and the leak went away.

This is what I was using:

http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
sslflags=NO_DEFAULT_CA
http_port localhost:3128
ssl_bump bump all
bump all is useless without peek/splice.

Let's see on my config snippets:

https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt key=/usr/local/squid/etc/rootCA2.key tls-cafile=/usr/local/squid/etc/rootCA12.crt options=SINGLE_DH_USE:SINGLE_ECDH_USE tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt key=/usr/local/squid/etc/rootCA2.key tls-cafile=/usr/local/squid/etc/rootCA12.crt options=SINGLE_DH_USE:SINGLE_ECDH_USE tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
# Cert database on ramdisk
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 1GB
sslcrtd_children 32 startup=10 idle=5

# SSL bump rules
acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex "/usr/local/squid/etc/acl.url.nobump"
ssl_bump peek DiscoverSNIHost
ssl_bump splice NoSSLIntercept
ssl_bump bump all


sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
sslcrtd_children 32 startup=2 idle=2
This is defaults. Pay attention, -M is limits use ssl_db directory to 4 Mb in size. It's too few for production servers. My ramdisk for ssl db is 1+ Gb in size:

/dev/ramdisk/ramdisk1           961M   14M  890M   2% /ramdisk1/ssl_db

sslproxy_session_cache_size 100 MB
This is disbalanced size instead of previous setting. Why so big?

#  TAG: sslproxy_session_cache_size
#        Sets the cache size to use for ssl session
#Default:
# sslproxy_session_cache_size 2 MB

sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
NEVER use this options. It is unsafe.

SSL Bump is dangerous enough itself. Don't do it more unsafe additionally by yourself.

This was on a machine (EC2 VM) with 14GB of RAM.
Pay attention on several places:

1. OS memory allocator.
2. OpenSSL version.
3. OS configuration (IPC, shared memory, swap - all memory related).
4. Squid's memory/pools configuration.

Don't forget about: Often memory fragmentation seems like leaks. But no leaks occurs indeed.

Also, don't forget - squid's memory consumption is not only cache_mem, but also caching on-disk metadata (swap.state), pools settings, working memory areas, processes memory. And - also - such things like content adaptation (did you know wide uses ecap gzip adapter is leaky itself?).

But this is just for example.

In any case, dig to the OpenSSL/OS side. Squid's memory in most cases is ok.

I know, this appears SSL Bump is leaky. But this is not correct.

--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:47 AM, Yuri <yvoinov@xxxxxxxxx> wrote:
I've used it on all versions starting from 3.4.

Now I'm using Squid 5.0.0.

I'm afraid, my config is completely useless, because of it contains tons
of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
with different memory allocator for custom infrastructure.

You can't just take my config, implement it and hope it will give same
results for you.

At least, it uses non-system CA bundle, platform-specific configuration
parameters combinations, etc.

I can say, than SSL Bump is not directly related to memory leaks. Squid
itself almost not contains memory leaks now. Usually misconfiguration
leads to memory overhead.

As a recommendation, I can give some advices.

1. Use server with enough RAM. 4 Gb usually enough just for default
squid configuration. Usually whole system RAM usage should never be
bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
should always be free during normal running. This prevents OS allocator
pressure to your proxy and, also, increasing performance of proxy). In
case of medium proxy server 16 Gb of RAM seems big enough, but never try
to fill it up completely.

2. Don't set giant cache_mem. Remember how you platform allocates whole
RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
memory-related settings.

3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.

4. Never remember - SSL Bump increases your cache memory pressure due to
increasing caching. So, you still require to have enough memory in your
system.


13.03.2018 22:25, Aaron Turner пишет:
What version are you using Yuri?  Can you share your config?
Everytime I use ssl bump, I have massive memory leaks.  It's been
effectively unusable for me.
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:10 AM, Yuri <yvoinov@xxxxxxxxx> wrote:
Moreover,

SSL Bump combines with interception/explicit proxy in one setup.

And works perfectly.


13.03.2018 21:14, Marcus Kool пишет:
"SSL bump" is the name of a complex Squid feature.
With ssl_bump ACLs one can decide which domains can be 'spliced' (go
through the proxy untouched) or can be 'bumped' (decrypted).

Interception is not a requirement for SSL bump.

Marcus

On 13/03/18 11:44, Danilo V wrote:
I mean SSL bump in explicit mode.
So intercept is a essencial requirement for running SSL bump?

Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
<uhlar@xxxxxxxxxxx <mailto:uhlar@xxxxxxxxxxx>> escreveu:

    On 13.03.18 13:44, Danilo V wrote:
     >Is it possible/feasible to configure squid in explicit mode
with ssl
     >intercept?

    explicit is not intercept, intercept is not explicit.

    explicit is where browser is configured (manually or
automatically via WPAD)
    to use the proxy.

    intercept is where network device forcifully redirects http/https
connections
    to the proxy.

    maybe you mean SSL bump in explicit mode?

     >Due to architecture of my network it is not possible to implement
     >transparent proxy.

    excuse me?
    by "transparent" people mean what we usually call "intercept".

     >What would be the behavior of applications that dont support
proxy - i.e.
     >dont forward requests to proxy?

    they mest be intercepted.

    --
    Matus UHLAR - fantomas, uhlar@xxxxxxxxxxx
<mailto:uhlar@xxxxxxxxxxx> ; http://www.fantomas.sk/
    Warning: I wish NOT to receive e-mail advertising to this address.
    Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
    Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
    _______________________________________________
    squid-users mailing list
    squid-users@xxxxxxxxxxxxxxxxxxxxx
<mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx>
    http://lists.squid-cache.org/listinfo/squid-users



_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



-- 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux