On Fri, Jan 10, 2025 at 5:39 PM Jonathan Lee <jonathanlee571@xxxxxxxxx> wrote:
Thanks for the replyWhat OS are you using? How many CPU cores do you want to dedicate to Squid? How much memory?I am using FreeBSD variant 4GB ram 2 CPUs pfSense plus
Starting CPU 1 (1) FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpulist0: <Open Firmware CPU Group> on ofwbus0 cpu0: <Open Firmware CPU> on cpulist0 cpu1: <Open Firmware CPU> on cpulist0 e6000sw0: CPU port at 5 CPU 0: ARM Cortex-A53 r0p4 affinity: 0 CPU 1: ARM Cortex-A53 r0p4 affinity: 1 I do not know how to dedicate specific CPU cores to Squid I do not think I can with pfSense plus.
It's more about how many squid workers to start. Then the OS will do the allocation
What filesystem are you using? For modern filesystems (ext4, btrfs, apfs) this parameter is much less meaningful than 10 years ago as they store directories as trees instead of lists.=> 1 250069679 ada0 MBR (119G) 1 532480 1 efi (260M) 532481 131072 2 fat32 (64M) 663553 249406127 3 freebsd [active] (119G) => 0 249406127 ada0s3 BSD (119G) 0 16 - free - (8.0K) 16 235528175 1 freebsd-zfs (112G) 235528191 13877248 2 freebsd-swap (6.6G) 249405439 688 - free - (344K) => 40 500118112 nda0 GPT (238G) 40 2008 - free - (1.0M) 2048 16777216 1 freebsd-swap (8.0G) 16779264 482344960 2 freebsd-ufs (230G) 499124224 993928 - free - (485M)ada0 is for the host osnda0 is my cache is uses freebsd-ufs I use the command mount_msdosfs /dev/nda0p2 /nvme/LOGS_Octane
huh? From the output above, it would seem that nda0 is using FreeBD UFS, not msdos.
Might be related to the partition table type you're using? I'm not a freebsd expert tho
The only way to mount the NVMe drive I have found is with mount_msdosfs maybe this causes a slow down I do not know but I can write and save to the drive this way. it is on nda0p2 230GBGeom name: nda0 modified: false state: OK fwheads: 255 fwsectors: 63 last: 500118151 first: 40 entries: 128 scheme: GPT Providers: 1. Name: nda0p1 Mediasize: 8589934592 (8.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 1048576 Mode: r1w1e2 efimedia: HD(1,GPT,04d31fb2-c0fd-11ef-8536-90ec770dda25,0x800,0x1000000) rawuuid: 04d31fb2-c0fd-11ef-8536-90ec770dda25 rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swapUSB length: 8589934592 offset: 1048576 type: freebsd-swap index: 1 end: 16779263 start: 2048 2. Name: nda0p2 Mediasize: 246960619520 (230G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 8590983168 Mode: r1w1e1 efimedia: HD(2,GPT,d84dfc00-cb1c-11ef-afd9-90ec770dda25,0x1000800,0x1cc00000) rawuuid: d84dfc00-cb1c-11ef-afd9-90ec770dda25 rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b label: LOG length: 246960619520 offset: 8590983168 type: freebsd-ufs index: 2 end: 499124223 start: 16779264 Consumers: 1. Name: nda0 Mediasize: 256060514304 (238G) Sectorsize: 512 Mode: r2w2e5my memory replacement policy is LRU for memory cache it seems to run better with that, my options for memory replacement policy are HEAP GDSF I assume any HEAP will require more memory, HEAP LFUDF, HEAP LRU and just LRU.I think so but shouldn't be significantly moreShould I change Memory Replacement Policy from LRU to HEAP LRU? I have tried every one again there is also the Cache Replacement Policy: Currently set to HEAP LFUDA Should memory replacement policy and cache replacement policy both be the same, and or does one cause any performance issues with the other?
They should not be too different in terms of memory useage.
Current Config# This file is automatically generated by pfSense # Do not edit manually ! http_port 192.168.1.1:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE http_port 127.0.0.1:3128 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE https_port 127.0.0.1:3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE icp_port 0 digest_generation off dns_v4_first on pid_filename /var/run/squid/squid.pid cache_effective_user squid cache_effective_group proxy error_default_language en icon_directory /usr/local/etc/squid/icons visible_hostname Lee_Family.home.arpa cache_mgr jonathanlee571@xxxxxxxxx access_log /nvme/LOGS_Optane/Squid_Logs/access.log cache_log /nvme/LOGS_Optane/Squid_Logs/cache.log cache_store_log none netdb_filename /nvme/LOGS_Optane/Squid_Logs/netdb.state pinger_enable on pinger_program /usr/local/libexec/squid/pinger sslcrtd_program /usr/local/libexec/squid/security_file_certgen -s /var/squid/lib/ssl_db -M 4MB -b 2048 tls_outgoing_options cafile=/usr/local/share/certs/ca-root-nss.crt tls_outgoing_options capath=/usr/local/share/certs/ tls_outgoing_options options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE tls_outgoing_options cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS sslcrtd_children 10 logfile_rotate 10 debug_options rotate=10 shutdown_lifetime 3 seconds # Allow local network(s) on interface(s) acl localnet src 192.168.1.0/27 forwarded_for delete via off httpd_suppress_version_string on uri_whitespace strip acl block_hours time 00:30-05:00 ssl_bump terminate all block_hours http_access deny all block_hours icp_port 0 htcp_port 0 snmp_port 0 icp_access deny all htcp_access deny all snmp_access deny all acl getmethod method GET acl to_ipv6 dst ipv6 acl from_ipv6 src ipv6 #tls_outgoing_options cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls_outgoing_options options=NO_SSLv3,NO_TLSv1,NO_TLSv1_1,NO_TICKET,SINGLE_DH_USE,SINGLE_ECDH_USE #tls_outgoing_options default-ca=on acl HttpAccess dstdomain '/usr/local/pkg/http.access' acl windowsupdate dstdomain '/usr/local/pkg/windowsupdate' #acl rewritedoms dstdomain '/usr/local/pkg/desdom' #store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt #store_id_children 10 startup=5 idle=1 concurrency=0 #always_direct allow all #store_id_access deny connect #store_id_access deny !getmethod #store_id_access allow rewritedoms #store_id_access deny all refresh_all_ims on reload_into_ims on max_stale 20 years minimum_expiry_time 0 #refresh_pattern -i ^http.*squid.internal.* 43200 100% 79900 override-expire override-lastmod ignore-reload ignore-no-store ignore-must-revalidate ignore-private ignore-auth refresh_pattern -i windowsupdate.com/.*.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims refresh_pattern -i microsoft.com/.*.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims refresh_pattern -i windows.com/.*.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims refresh_pattern -i microsoft.com.akadns.net/.*.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims refresh_pattern -i deploy.akamaitechnologies.com/.*.(cab|exe|ms[i|u|f|p]|[ap]sf|wm[v|a]|dat|zip|psf) 43200 80% 129600 reload-into-ims acl https_login url_regex -i ^https.*(login|Login).* cache deny https_login #range_offset_limit 512 MB windowsupdate range_offset_limit 0 !windowsupdate quick_abort_min -1 KB cache_mem 64 MB maximum_object_size_in_memory 256 KB memory_replacement_policy lru cache_replacement_policy heap LFUDA minimum_object_size 0 KB maximum_object_size 512 MB cache_dir aufs /nvme/LOGS_Optane/Squid_Cache 32000 16 256 offline_mode off cache_swap_low 90 cache_swap_high 95 acl donotcache dstdomain '/var/squid/acl/donotcache.acl' cache deny donotcache cache allow all # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|?) 0 0% 0 refresh_pattern . 0 20% 4320 #Remote proxies # Setup some default acls # ACLs all, manager, localhost, and to_localhost are predefined. acl allsrc src all acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 8080 3128 3129 1025-65535 acl sslports port 443 563 8080 5223 2197 acl purge method PURGE acl connect method CONNECT # Define protocols used for redirects acl HTTP proto HTTP acl HTTPS proto HTTPS # SslBump Peek and Splice # http://wiki.squid-cache.org/Features/SslPeekAndSplice # http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit # Match against the current step during ssl_bump evaluation [fast] # Never matches and should not be used outside the ssl_bump context. # # At each SslBump step, Squid evaluates ssl_bump directives to find # the next bumping action (e.g., peek or splice). Valid SslBump step # values and the corresponding ssl_bump evaluation moments are: # SslBump1: After getting TCP-level and HTTP CONNECT info. # SslBump2: After getting TLS Client Hello info. # SslBump3: After getting TLS Server Hello info. # These ACLs exist even when 'SSL/MITM Mode' is set to 'Custom' so that # they can be used there for custom configuration. acl step1 at_step SslBump1 acl step2 at_step SslBump2 acl step3 at_step SslBump3 acl banned_hosts src '/var/squid/acl/banned_hosts.acl' acl blacklist dstdom_regex -i '/var/squid/acl/blacklist.acl' http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !safeports http_access deny CONNECT !sslports # Always allow localhost connections http_access allow localhost quick_abort_min 0 KB quick_abort_max 0 KB quick_abort_pct 95 request_body_max_size 0 KB delay_pools 1 delay_class 1 2 delay_parameters 1 -1/-1 -1/-1 delay_initial_bucket_level 100 delay_access 1 allow allsrc # Reverse Proxy settings deny_info TCP_RESET allsrc # Package Integration url_rewrite_program /usr/local/bin/squidGuard -c /usr/local/etc/squidGuard/squidGuard.conf url_rewrite_bypass off url_rewrite_children 25 startup=12 idle=8 concurrency=0 # Custom options before auth #host_verify_strict on # These hosts are banned http_access deny banned_hosts # Block access to blacklist domains http_access deny blacklist # List of domains allowed to logging in to Google services request_header_access X-GoogApps-Allowed-Domains deny all request_header_add X-GoogApps-Allowed-Domains consumer_accounts # Set YouTube safesearch restriction acl youtubedst dstdomain -n www.youtube.com m.youtube.com youtubei.googleapis.com youtube.googleapis.com www.youtube-nocookie.com request_header_access YouTube-Restrict deny all request_header_add YouTube-Restrict none youtubedst # Custom SSL/MITM options before auth acl wpad urlpath_regex ^/wpad.dat$ acl wpad urlpath_regex ^/proxy.pac$ acl wpad urlpath_regex ^/wpad.da$ deny_info TCP_RESET wpad #deny_info 200:/etc/squid/wpad.dat wpad reply_header_access Content-Type deny wpad http_access deny wpad http_access deny !safeports http_access deny CONNECT !sslports cachemgr_passwd disable offline_toggle reconfigure shutdown cachemgr_passwd redacted all eui_lookup on acl no_miss url_regex -i gateway.facebook.com/ws/realtime? acl no_miss url_regex -i web-chat-e2ee.facebook.com/ws/chat acl CONNECT method CONNECT acl wuCONNECT dstdomain www.update.microsoft.com acl wuCONNECT dstdomain sls.microsoft.com http_access allow CONNECT wuCONNECT localnet http_access allow CONNECT wuCONNECT localhost http_access allow CONNECT windowsupdate localnet http_access allow CONNECT windowsupdate localhost http_access allow CONNECT HttpAccess localnet http_access allow CONNECT HttpAccess localhost http_access deny to_ipv6 http_access deny from_ipv6 acl BrokenButTrustedServers dstdomain '/usr/local/pkg/dstdom.broken' acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch sslproxy_cert_error deny all acl splice_only_ip src 192.168.1.8 acl splice_only_ip src 192.168.1.10 acl splice_only_ip src 192.168.1.11 acl splice_only_ip src 192.168.1.15 acl splice_only_ip src 192.168.1.16 :::: = redacted mac address acl splice_only_mac arp ::::: acl splice_only_mac arp ::::: acl splice_only_mac arp ::::: acl splice_only_mac arp ::::: acl splice_only_mac arp ::::: acl NoSSLIntercept ssl::server_name_regex -i '/usr/local/pkg/reg.url.nobump' acl NoBumpDNS dstdomain -n '/usr/local/pkg/dns.nobump' acl SSL_Intercept_Terminate dstdomain -n '/usr/local/pkg/url.bump' acl active_use annotate_client active=true acl bump_only_ip src 192.168.1.3 acl bump_only_ip src 192.168.1.4 acl bump_only_ip src 192.168.1.5 #acl bump_only_ip src 192.168.1.6 acl bump_only_ip src 192.168.1.9 acl bump_only_ip src 192.168.1.13 acl bump_only_mac arp ::::: acl bump_only_mac arp ::::: acl bump_only_mac arp ::::: acl bump_only_mac arp ::::: acl bump_only_mac arp ::::: #acl bump_only_mac arp ::::: coredump_dir /nvme/LOGS_Optane/Squid_Dump acl splice_group any-of https_login NoBumpDNS NoSSLIntercept acl splice_only_local_group all-of splice_only_mac splice_only_ip acl splice_main any-of splice_group splice_only_local_group acl bump_main all-of bump_only_mac bump_only_ip ssl_bump peek step1 ssl_bump terminate SSL_Intercept_Terminate miss_access deny no_miss active_use ssl_bump splice splice_main active_use ssl_bump bump bump_main active_use acl activated note active_use true ssl_bump terminate !activated # Setup allowed ACLs # Allow local network(s) on interface(s) http_access allow localnet # Default block all to be sure http_access deny allsrcDoes delay pool setting cause any issues? They seem to be default values one pool.
If you need them, you need them.
There doesn't seem to be anything obviously wrong with the highlighted parts of your config, except maybe that the memory cache is tiny.
How much RAM is the OS reporting that squid is using after a few hours or days of use?
Francesco
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx https://lists.squid-cache.org/listinfo/squid-users
