Re: CPU Load 100% after implementing SSL Bump ....

[Sagar Malve <sagarmalve91@xxxxxxxxx>]

Hi Team,

Squid - Version 3.5.13

Please find the below Squid Cache Logs
2016/05/23 13:35:55 kid1| Error negotiating SSL connection on FD 138: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:35:55 kid1| Error negotiating SSL connection on FD 457: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:00 kid1| Error negotiating SSL connection on FD 33: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:01 kid1| Error negotiating SSL connection on FD 438: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:05 kid1| Error negotiating SSL connection on FD 555: (104) Connection reset by peer
2016/05/23 13:36:06 kid1| Error negotiating SSL connection on FD 512: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:09 kid1| Error negotiating SSL connection on FD 618: (104) Connection reset by peer
2016/05/23 13:36:15 kid1| Error negotiating SSL connection on FD 514: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:15 kid1| Error negotiating SSL connection on FD 206: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown (1/0)
2016/05/23 13:36:18 kid1| Error negotiating SSL connection on FD 627: (104) Connection reset by peer
2016/05/23 13:36:18 kid1| Error negotiating SSL on FD 147: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (1/0/0)
2016/05/23 13:36:19 kid1| Error negotiating SSL connection on FD 343: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown (1/0)
2016/05/23 13:36:24 kid1| Error negotiating SSL connection on FD 378: (104) Connection reset by peer
2016/05/23 13:36:25 kid1| Error negotiating SSL connection on FD 491: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:28 kid1| ctx: enter level  0: 'http://afs.moatads.com/empty_flash?tracer='
2016/05/23 13:36:28 kid1| keepaliveAccounting: Impossible keep-alive header from 'http://afs.moatads.com/empty_flash?tracer='
2016/05/23 13:36:34 kid1| ctx: exit level  0
2016/05/23 13:36:34 kid1| Error negotiating SSL connection on FD 257: (104) Connection reset by peer
2016/05/23 13:36:34 kid1| Error negotiating SSL connection on FD 90: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:38 kid1| Error negotiating SSL on FD 125: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (1/0/0)
2016/05/23 13:36:38 kid1| Error negotiating SSL connection on FD 577: (104) Connection reset by peer
2016/05/23 13:36:38 kid1| Error negotiating SSL connection on FD 91: (104) Connection reset by peer
2016/05/23 13:36:39 kid1| Error negotiating SSL connection on FD 220: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:43 kid1| Error negotiating SSL connection on FD 50: (104) Connection reset by peer
2016/05/23 13:36:48 kid1| Error negotiating SSL connection on FD 579: (104) Connection reset by peer
2016/05/23 13:36:48 kid1| Error negotiating SSL connection on FD 455: (104) Connection reset by peer
2016/05/23 13:36:49 kid1| Error negotiating SSL connection on FD 414: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:39:28 kid1| varyEvaluateMatch: Oops. Not a Vary match on second attempt, 'http://cdn.sstatic.net/Sites/stackoverflow/all.css?v=fada5080e3ea' 'accept-encoding="gzip,%20deflate"'

----------------------------Cache log End --------------------------------------

Do we need to update openssl? I got to know these from the forum previous post ....

If we need to update the openssl then where can we find the updated version of CA Certs ....

On Mon, May 23, 2016 at 12:52 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote:

On 23/05/2016 6:27 p.m., Sagar Malve wrote:
> Hi Team,
>
> System Config:
>
> Intel S2400SC2 Motherboard
> Intel Xeon ES 2407 V2 CPU
> RAM 32 GB
>

What Squid version?

>
> http_port 3127
> http_port 3128 intercept
> https_port 3129 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
> key=/etc/squid/ssl_certs/squid.key options=NO_SSLv3
> tls-dh=/etc/squid/dhparam.pem
> sslproxy_capath /etc/ssl/certs
>
>
> # FILTERING HTTPS
> acl 1 dstdomain .fbcdn.net .akamaihd.net .fbsbx.com
> #acl 2a dstdomain .mahadana.com .mql4.com .metaquotes.net
> acl 2 url_regex -i ^https?:\/\/attachment\.fbsbx\.com\/.*\?(id=[0-9]*).*
> acl 2 url_regex -i
> \.fbsbx\.com\/.*\/(.*\.(unity3d|pak|zip|exe|dll|jpg|png|gif|swf)/)$
> acl 2 url_regex -i ^https?:\/\/.*\.ytimg\.com(.*\.(webp|jpg|gif))
> acl 2 url_regex -i ^https?:\/\/([^\.]*)\.yimg\.com\/(.*)
> acl 2 url_regex -i ^https?:\/\/.*\.gstatic\.com\/images\?q=tbn\:(.*)
> acl 2 url_regex -i
> ^https?:\/\/.*\.reverbnation\.com\/.*\/(ec_stream_song|download_song_direct|stream_song)\/([0-9]*).*
> acl 2 url_regex -i
> ^https?:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|.exoclick\.com|interclick.\com|\.googlesyndication\.com|\.auditude\.com|.visiblemeasures\.com|yieldmanager|cpxinteractive)(.*)
> acl 2 url_regex -i ^https?:\/\/(.*?)\/(ads)\?(.*?)
> acl 2 url_regex -i ^https?:\/\/.*steampowered\.com\/.*\/([0-9]+\/(.*))
> acl 3 url_regex -i
> ^https?:\/\/(.*?)\/speedtest\/.*\.(jpg|txt|png|gif|swf)\?.*
> acl 3 url_regex -i speedtest\/.*\.(jpg|txt|png|gif|swf)\?.*
> acl 4 url_regex -i reverbnation.*audio_player.*ec_stream_song.*$
> acl 5 url_regex -i utm.gif.*
> acl 6 url_regex -i c.android.clients.google.com.market.GetBinary.GetBinary.*
> acl 7 url_regex -i youtube.*(ptracking|stream_204|player_204|gen_204).*$
> acl 7 url_regex -i
> \.c\.(youtube|google)\.com\/(get_video|videoplayback|videoplay).*$
> acl 7 url_regex -i (youtube|google).*\/videoplayback\?.*
> acl 8 http_status 302
> acl getmethod method GET
>

Using .* on the beginning or end of a regex does nothing but cause more
CPU workload for Squid.

If you put it inside (.*), or with an anchor ^.* or .*$ just makes the
CPU usage worse.

What http_access rules are using those?

>
> ssl_bump splice localhost
> acl 9 at_step SslBump1
> acl 10 at_step SslBump2
> acl 11 at_step SslBump3
> ssl_bump peek 9 all
> ssl_bump bump 10 all
> ssl_bump bump 11 all

Step3 of bumping process will never happen. You told Squid to begin
decryption at step2.

Have you disabled "via"?


>
> ----------------------------------------------------------------------------------------------
>
> Is there any way where it can Cache SSL Certificate for all HTTPS Traffic
> ....
> Because SSL Cert & Squid process were using 99% of CPU Load ....

Er, what do you think caching does exactly?

Caching HTTPS will have no effect on your described CPU problem. Might
make it worse even.


Between them?

How much is each process using?

How may concurrent connections are being handled by Squid to get that
loading ?


Check whether Squid is finished loading its cache_dir indexes, or if any
of them are undergoing a "DIRTY" rebuild. That can use a lot of CPU
while its happening and caching cannot be fully operational until its
finished either.


>
> We have approx 200 users ....
>
> I have set the open file limit to 100000

FYI: SSL-Bump with your configuration will use 3 FD for each client
inbound HTTPS request. That 100K limit will restrict your users to 150
concurrent connections each.
A browser using Happy eyeballs will open 16 connections to each domain.
Average web page on the most popular sites involve around 100 objects
spread over 10+ domains.
  => ~160 FD needed to load an average page.

I'd double that limit, if you expect this proxy to have much traffic.

>
> Could you please let us know if there is any way to Cache the HTTPS Request
> in Squid .....
>

You are already SSL-Bumping traffic. That removes the 'S' from HTTPS.
Leaving Squid with regular HTTP messages, which already are cached if it
can.

Amos

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users