Re: squid logging disable based on ACL & kernel: Out of memory

[Akshay Hegde <akshay.k.hegde@xxxxxxxxx>]

Hi Alex,

I updated to latest squid as you suggested, and I tried SSL-Bump using below config (which filters URLs which are in 443 too), however I have 600 users (windows, linux, Mac, mobile OS like Androd, Windows etc), so asking them to import CA certificate in browser is not feasible.

1. Is there any way to filter HTTPS URLs without importing CA certificates on client side? if available can you share config snippet

2. for 16GB RAM, 4 core CPU, 8GB Swap, expected to have 10GB cache,  how to calculate configurations parameters, is there any thumb rule ? please share how you usually calculate.

# config

cache_mgr webmaster
cache deny QUERY
cache_mem 256 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4 MB
minimum_object_size 0 KB
maximum_object_size_in_memory 512 kB
ipcache_size 2048
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
cache_replacement_policy lru
memory_replacement_policy lru
cache_dir ufs /var/spool/squid 10000 16 256
cache_effective_user squid
cache_effective_group squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
memory_pools on
memory_pools_limit 5 MB

# SSL-Bump -working but not feasible.

http_port 3128 ssl-bump cert=/etc/squid/sslcert/proxyCA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/lib64/squid/security_file_certgen -s  /var/spool/squid/ssl_db -M 4MB
sslcrtd_children 5
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

------------------------------------ My New Environment --------------------

# squid -v
Squid Cache: Version 4.4
Service Name: squid

# cat /etc/redhat-release
CentOS Linux release 8.1.1911 (Core)

# Tested ACLs

logformat test_log %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %>ru %[un %Sh/%<a %mt
acl test_sites dstdomain "/etc/squid/acls/test_sites.acl"
access_log /var/log/squid/test_site.log test_log test_sites

# tail -f /var/log/squid/test_site.log
1588678050.178   3247 10.0.2.15 TCP_TUNNEL/200 28073 CONNECT nav.sciencedirect.com:443 akshay HIER_DIRECT/91.235.133.74 -
1588678050.189   3942 10.0.2.15 TCP_TUNNEL/200 24000 CONNECT nav.sciencedirect.com:443 akshay HIER_DIRECT/91.235.133.74 -
1588678050.355   2552 10.0.2.15 TCP_TUNNEL/200 788 CONNECT nav.sciencedirect.com:443 akshay HIER_DIRECT/91.235.133.74 -

1588681419.635    647 10.0.2.15 TCP_MISS/200 402 POST http://scratchpads.eu/modules/statistics/statistics.php akshay HIER_DIRECT/157.140.2.32 text/html
1588681420.055   1069 10.0.2.15 TCP_MISS/200 46772 GET http://scratchpads.eu/sites/all/themes/scratchpads_eu/images/shrimp-202px.png akshay HIER_DIRECT/157.140.2.32 image/png

On Sat, May 2, 2020 at 1:00 AM Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> wrote:

On 5/1/20 12:43 PM, Akshay Hegde wrote:

> I have below option globally, which I don't want to make "off"
> strip_query_terms on

> acl track dstdomain "/etc/squid/sites_track.txt"
> access_log /var/log/squid/full_site_links.log squid_custom track

> however for specific ACL I would like to log full URL with query
> parameters, how this can be done ?

I have not tested this, and the results may be version-dependent, but
according to logformat documentation[1], %ru honors strip_query_terms
while %>ru does not:

    logformat strippedFormat %ts... %ru ...
    access_log ... strippedFormat track !specific_ACL

    logformat detailedFormat %ts... %>ru ...
    access_log ... detailedFormat track specific_ACL

[1] http://www.squid-cache.org/Doc/config/logformat/


HTH,

Alex.

> On Fri, May 1, 2020 at 7:05 PM Alex Rousskov wrote:
>
>     On 5/1/20 1:20 AM, Akshay Hegde wrote:
>
>     > *1. How to disable logging of few ACLs ?
>
>     Use "access_log none aclX" to prevent creation of access.log records for
>     transactions matching aclX. See
>     http://lists.squid-cache.org/pipermail/squid-users/2020-April/021876.html
>     for
>     some related caveats.
>
>
>     > *2. Kernel Out of Memory
>
>     This problem is most likely unrelated to logging. If your Squid is
>     gradually leaking memory (rather than just being overwhelmed with
>     traffic), then the first step towards removing those memory leaks would
>     be to upgrade your Squid from the unsupported and buggy v3.1.10.
>
>
>     HTH,
>
>     Alex.
>
>
>
> --
> <https://about.me/akshay.k.hegde?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=thumb>
>       
> Akshay Hegde
> about.me/akshay.k.hegde
> <https://about.me/akshay.k.hegde?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=thumb>
>
>

--

Akshay Hegde about.me/akshay.k.hegde

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users