Hi all, I tried with stock squid from Ubuntu 18.04 (version 4.10) and basic config, but still no luck. Here is the config I tried: ----------------------------------------- acl localnet src 10.20.0.0/16 # My network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT cache deny all http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access allow localnet manager http_access deny manager http_access deny to_localhost http_access allow localnet http_access allow localhost http_access deny all http_port localhost:3128 http_port 10.20.6.8:50505 coredump_dir /var/spool/squid shutdown_lifetime 2 seconds max_filedescriptors 262143 server_persistent_connections off client_persistent_connections off request_header_access Via deny all request_header_access X-Forwarded-For deny all dns_v4_first on ------------------------------------ Looks like squid has some serious memory issues when under heavy load (90 servers that crawl Internet sites). It just eats up memory, and does not free it up even days after it is being used (with no load on the proxy for days). So I guess I have to look for another solution. Best regards, Ivan Bulatovic On Mon, Jul 20, 2020 at 10:46 PM Ivan Bulatovic <ivan.bulatovic@xxxxxxxxx> wrote: > > Hi all, > > I am trying to configure squid to run as a forward proxy with no > caching (cache deny all) with an option to choose the outgoing IP > address based on the username. So all squid has to do is to use a > certain outgoing IP address for a certain user, return the data from > the server to that user and cache nothing. > > For that I created a special authentication helper and used the ACLs > and tcp_outgoing_address to create a lot of users and outgoing IP > addresses (about 260 at the moment). Example (not the real IP I use, > of course): > > acl use_IP1 proxy_auth user1 > tcp_outgoing_address 1.2.3.4 use_IP1 > > I also configured the squid to use 4 workers, but this happens even > when I use only one worker (default) > > And this works. However, under heavy load, Squid eats all of the RAM > and then starts going to swap. And the memory usage does not drop when > I remove all the load from squid (I shut down all clients). > > I left it to see if the memory will be freed but even after leaving it > for an hour the info page reports this: > Cache information for squid: > Hits as % of all requests: 5min: 0.0%, 60min: 0.0% > Hits as % of bytes sent: 5min: 0.0%, 60min: 1.1% > Memory hits as % of hit requests: 5min: 0.0%, 60min: 0.0% > Disk hits as % of hit requests: 5min: 0.0%, 60min: 100.0% > Storage Swap size: 0 KB > Storage Swap capacity: 0.0% used, 100.0% free > Storage Mem size: 0 KB > Storage Mem capacity: 0.0% used, 100.0% free > Mean Object Size: 0.00 KB > Requests given to unlinkd: 0 > > Resource usage for squid: > UP Time: 255334.875 seconds > CPU Time: 7122.436 seconds > CPU Usage: 2.79% > CPU Usage, 5 minute avg: 0.05% > CPU Usage, 60 minute avg: 37.66% > Maximum Resident Size: 41500720 KB > Page faults with physical i/o: 1003410 > > And here is the listing of free and top commands (with no load on the server): > > # free -h > total used free shared buff/cache available > Mem: 11G 10G 791M 676K 491M 1.0G > Swap: 11G 5.5G 6.5G > > # top > top - 14:12:32 up 3 days, 1:30, 1 user, load average: 0.00, 0.00, 0.00 > Tasks: 177 total, 1 running, 102 sleeping, 0 stopped, 0 zombie > %Cpu0 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu1 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu2 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu3 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu4 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu5 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu6 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > %Cpu7 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st > KiB Mem : 91.2/12251688 > [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| > ] > KiB Swap: 45.8/12582904 > [|||||||||||||||||||||||||||||||||||||||||||||| > ] > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 7851 proxy 20 0 6946872 2.514g 8084 S 0.0 21.5 29:43.74 squid > 7832 proxy 20 0 6711480 2.464g 8040 S 0.0 21.1 29:58.17 squid > 7814 proxy 20 0 6834928 2.454g 10024 S 0.0 21.0 29:47.56 squid > 7843 proxy 20 0 6906252 2.436g 8208 S 0.0 20.8 29:15.60 squid > 1329 root 20 0 2416672 167272 12680 S 0.0 1.4 136:18.57 metricbeat > 1321 root 20 0 1831804 48364 11648 S 0.0 0.4 14:32.10 filebeat > 474 root 19 -1 127796 17576 17144 S 0.0 0.1 0:27.01 > systemd-journal > 7811 proxy 20 0 549384 14168 8372 S 0.0 0.1 0:20.87 squid > 1166 root 20 0 1749724 10596 4468 S 0.0 0.1 0:31.83 snapd > 43940 proxy 20 0 28884 9608 5384 S 0.0 0.1 0:00.14 python3 > 43941 proxy 20 0 28884 9552 5328 S 0.0 0.1 0:00.10 python3 > 43939 proxy 20 0 28884 9524 5308 S 0.0 0.1 0:00.12 python3 > 43938 proxy 20 0 28884 9452 5232 S 0.0 0.1 0:00.16 python3 > 48848 root 20 0 105688 6960 5968 S 0.0 0.1 0:00.02 sshd > 48974 janitor 20 0 108120 5380 4372 S 0.0 0.0 0:00.00 sshd > 1 root 20 0 86360 4364 2488 S 0.0 0.0 32:46.22 systemd > ... > ... lines ommited > ... > > In the attachment you can find the printout from squidclient mgr:info > and squidclient mgr:mem. These are both taken at the moment when there > is no more load on the proxy. I also included my squid.conf file > (minus the two files where acls are defined and outgoing IP addresses, > these two contain only acl and tcp_outgoing_address lines as in the > example above). > > Machine info: > OS: Ubuntu 18.04 with latest updates > Squid version 4.12 (from diladele repository) > Hardware: Hyper-V virtual machine with 8 vCPU, 12GB of RAM > > I can not understand what is eating all of the memory, if I disabled the cache. > > Maybe I configured something wrong but I can not find what. > > Thank you for any help you can provide. > > Best regards, > Ivan _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
