Lets hope for the best. When you will be with 3.5.24 we would be able to handle things in a simpler manner. There are couple options to handle a situation like this and one of the options is to fence the ssl_crtd with some kind of lock mechanism for the DB rebuild time. For example a bash script that will run the ssl_crtd and will try to trace if there is an exit code else then 0. If such a case exists it will check for some lock file. If it doesn't exit it will create it and will send a healing request to another daemon or will start the self healing by itself. There are pros and cons but I believe that 3.5.24 will help more then many other options. All The Bests, Eliezer ---- http://ngtech.co.il/lmgtfy/ Linux System Administrator Mobile: +972-5-28704261 Email: eliezer@xxxxxxxxxxxx From: --Ahmad-- [mailto:ahmed.zaeem@xxxxxxxxxxxx] Sent: Friday, March 3, 2017 2:20 PM To: Yuri Voinov <yvoinov@xxxxxxxxx> Cc: Eliezer Croitoru <eliezer@xxxxxxxxxxxx>; squid-users@xxxxxxxxxxxxxxxxxxxxx Subject: Re: squid 3.5.2==> HTTPS FATAL: The ssl_crtd helpers are crashing too rapidly, need help! @ eliezer i was using children as 10 ans faced the problem so i trued to increase children to 1000 to see if this was the reason and unfortunately the same problem . ys I’m using debian 6 os . i appreciable the helping from all the replies below but so far i havent got any clear solution . now i updated to 3.5.24 last one . and will see it if comes back …i will update the list with result . if it failed … I’m forced to create cron job to remove the certs like every 24 hours . thank you guys all of you . thanks amos , thanks eliezer , thanks yuri kind regards On Mar 3, 2017, at 1:37 PM, Yuri Voinov <mailto:yvoinov@xxxxxxxxx> wrote: 03.03.2017 6:32, Eliezer Croitoru пишет: Hey Yuri, This issue is not 100% squid but I think it's related to the way ssl_crtd works. I am not sure if it has some locking or other things to prevent such issues. The first solution is to somehow defend the DB from corruption, like in a case that more then a dozen identical requests are being done towards a single site and two ssl_crtd helpers are trying to do the same things. I believe that something to fence this should already be inside squid and ssl_crtd but I am pretty sure this is the main issue. I suggests this can be external reason to occurs this issue. Somehow, for example, BlueCoat on ISP upstream, tcp packets corruption, etc. I dont know, just guessing. Alex and his team should know the answer for this subject and if I'm not wrong theoretically there are couple ways to prevent the mentioned issues. I had a plan to try and understand the ssl_crtd code and interface but yet to do so. I hope this issue will be resolved in a way that it can be backported to 3.5 in the worst case. I hope too, but if it external..... fewwwwwwwwwww. Anyway, watchdog is good backup to preventing manual interventions by SA. Eliezer ---- http://ngtech.co.il/lmgtfy/ Linux System Administrator Mobile: +972-5-28704261 Email: mailto:eliezer@xxxxxxxxxxxx From: squid-users [mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Yuri Voinov Sent: Thursday, March 2, 2017 11:46 PM To: mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx Subject: Re: squid 3.5.2==> HTTPS FATAL: The ssl_crtd helpers are crashing too rapidly, need help! This problem, in principle, is common to all versions of ssl-bumped Squid from version 3.4 and 5.0, inclusive, and occurs when the stored certificate is damaged for any reason. The only thing vorkeraund that I could find - a monitor kesh.log and initialize the certificate database again with squid restart automatically. In some installations, this problem does not occur over the years. In other - almost daily. I have no desire to find out why this is happening exactly. For me it was easier to make the watchdog, which will follow up on this. 03.03.2017 3:40, Yuri Voinov пишет: One hint finally: '([^ ]*) helper database ([^ ]*) failed: The SSL certificate database ([^ ]*) is corrupted. Please rebuild' - - - 0 exec "/usr/local/bin/crtd_create.sh -r >/dev/null 2>&1" 'FATAL: ([^ ]*) helpers are crashing too rapidly, need help!' - - - 0 exec "/usr/local/bin/crtd_create.sh -r >/dev/null 2>&1" 'Cannot add certificate to db.' - - - 0 exec "/usr/local/bin/crtd_create.sh -r >/dev/null 2>&1" PS. This is from logsurfer.conf. 03.03.2017 3:34, Yuri Voinov пишет: This error is usually preceded by another error in cache.log associated with the certificates. I will show you the direction. Then go himself. This software will useful for you to solve: http://www.crypt.gen.nz/logsurfer/ HTH, Yuri 03.03.2017 2:47, --Ahmad-- пишет: hey folks . i have a problem with squid it get crashed after i enabled https ! cache log error => FATAL: The ssl_crtd helpers are crashing too rapidly, need help! i googled many topics and relevant pages and couldnt find a clear solution . the quick solution i made was i removed the certs in file : rm -rfv /var/lib/ssl_db/ then reinitiated the DB using cmd below : /lib/squid/ssl_crtd -c -s /var/lib/ssl_db chown -R squid.squid /var/lib/ssl_db chown -R squid.squid /var/lib/ssl_db the restarted squid . but this is not a solution becuase squid get crashed again after certain time and i don’t know why ! my version is 3.5.2 here is squid.conf : /etc/squid/squid.conf visible_hostname pcloud acl ip1 myip 10.1.0.1 acl ip2 myip 192.168.10.210 tcp_outgoing_address 192.168.10.210 ip1 tcp_outgoing_address 192.168.10.210 ip2 # # Recommended minimum configuration: # # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # # Recommended minimum Access Permission configuration: # # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports http_access allow CONNECT # Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on "localhost" is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow localnet http_access allow localhost # And finally deny all other access to this proxy http_access deny all # Squid normally listens to port 3128 http_port 3128 # Uncomment and adjust the following to add a disk cache directory. #cache_dir ufs /var/cache/squid 100 16 256 # Leave coredumps in the first cache dir #coredump_dir /var/cache/squid # # Add any of your own refresh_pattern entries above these. # # http_port 3126 #http_port 3128 ####################################### #cache_swap_low 90 #cache_swap_high 95 ############################ cache_effective_user squid cache_effective_group squid memory_replacement_policy lru cache_replacement_policy heap LFUDA ######################## maximum_object_size 10000 MB #cache_mem 5000 MB maximum_object_size_in_memory 10 MB ######################### logfile_rotate 2 max_filedescriptors 131072 ############################### ############ cache_dir aufs /var/cache/squid 600000 64 128 ####################################### https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/usr/local/squid/ssl_cert/myca.pem key=/usr/local/squid/ssl_cert/myca.pem ssl_bump server-first all sslcrtd_program /lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB sslcrtd_children 1000 startup=1 idle=1 ### minimum_object_size 0 bytes #refresh patterns for caching static files refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i .(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i .(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i .(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i .index.(html|htm)$ 0 40% 10080 refresh_pattern -i .(html|htm|css|js)$ 1440 40% 40320 refresh_pattern . 0 40% 40320 any Joy Guys ? should i update squid ? or downgrade squid ? kind regards _______________________________________________ squid-users mailing list mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users -- Bugs to the Future <0x613DEC46.asc>_______________________________________________ squid-users mailing list mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
