On 4/14/21 2:49 PM, koshik moshik wrote: > First of all thank you for trying to help me. Let me describe my current > issue: I have 5000 proxies and would like to hide them. My plan was > using another proxy server with 5000 cache peers and 5000 users. Each > user would get one peer and one proxy attached to that peer. So > basically the outer world would not see my "main proxy" and only the one > from the new proxy server. > Is there any better solution than cache peers for that? Probably. It sounds like you need dynamic HTTP proxy routing (where the first proxy computes the address of the second proxy/hop from the user name rather than selects from a huge set of fixed cache_peers using a huge set of fixed ACLs). AFAIK, Squid does not support such routing without source code modifications. Even ICAP/eCAP cannot do that alone. Alex. > On Wed, Apr 14, 2021 at 8:37 PM Alex Rousskov wrote: > > On 4/14/21 2:29 AM, koshik moshik wrote: > > Thank you! Yes, it works fine with 5 peers. So, what would be the best > > solution to handle 5000 peers? > > As you can tell by other responses, you might be asking the wrong > question. However, I will still try to answer your question. The best > option may be to add support for a new Squid configuration parameter > that tells Squid to limit cache_peer candidate accumulation to N peers, > effectively making all those linear searches fast. > > https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F > <https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F> > > Alex. > > > > On Mon, Apr 12, 2021 at 6:03 PM Alex Rousskov wrote: > > > > On 4/10/21 5:03 PM, koshik moshik wrote: > > > > > I am trying to run a Squid proxy Server witth about 5000 cache > > peers. I > > > am running a dedicated server with 6 cores and 32GB RAM on > Ubuntu 16. > > > > > > > > > Could you tell me what else is needed / not needed in my > > squid.config? I > > > am encountering a high CPU usage and would like to create a very > > > efficient proxy server. > > > > IIRC, Squid code is not optimized for handling a large number of > > cache_peers: Several cache peer selection steps involve linear > searches. > > > > I do not know what exactly causes high CPU usage in your > environment but > > it could be those linear searches. You can test that > (indirectly) by > > decreasing the number of cache_peers from 5000 to, say, 5. > That is a > > weak test, of course, because other cache_peer-related > overheads could > > be to blame, but I would start there. > > > > > > HTH, > > > > Alex. > > > > > > > > > Down below you can find my squid.config(I deleted the other > cache_peer > > > lines): > > > > > > ----------- > > > > > > http_port 3128 > > > > > > dns_v4_first on > > > > > > acl SSL_ports port 1-65535 > > > > > > acl Safe_ports port 1-65535 > > > > > > acl CONNECT method CONNECT > > > > > > http_access deny !Safe_ports > > > > > > http_access deny CONNECT !SSL_ports > > > > > > auth_param basic program /usr/lib/squid/basic_ncsa_auth > > /etc/squid/.htpasswd > > > > > > auth_param basic children 5 > > > > > > auth_param basic realm Squid Basic Authentication > > > > > > auth_param basic credentialsttl 5 hours > > > > > > acl password proxy_auth REQUIRED > > > > > > http_access allow password > > > > > > #http_access deny all > > > > > > cache allow all > > > > > > never_direct allow all > > > > > > ident_access deny all > > > > > > > > > > > > > > > > > > cache_mem 1 GB > > > > > > maximum_object_size_in_memory 16 MB > > > > > > > > > > > > > > > > > > # Leave coredumps in the first cache dir > > > > > > coredump_dir /var/spool/squid > > > > > > > > > #Rules to anonymize http headers > > > > > > forwarded_for off > > > > > > request_header_access Allow allow all > > > > > > request_header_access Authorization allow all > > > > > > request_header_access WWW-Authenticate allow all > > > > > > request_header_access Proxy-Authorization allow all > > > > > > request_header_access Proxy-Authenticate allow all > > > > > > request_header_access Cache-Control allow all > > > > > > request_header_access Content-Encoding allow all > > > > > > request_header_access Content-Length allow all > > > > > > request_header_access Content-Type allow all > > > > > > request_header_access Date allow all > > > > > > request_header_access Expires allow all > > > > > > request_header_access Host allow all > > > > > > request_header_access If-Modified-Since allow all > > > > > > request_header_access Last-Modified allow all > > > > > > request_header_access Location allow all > > > > > > request_header_access Pragma allow all > > > > > > request_header_access Accept allow all > > > > > > request_header_access Accept-Charset allow all > > > > > > request_header_access Accept-Encoding allow all > > > > > > request_header_access Accept-Language allow all > > > > > > request_header_access Content-Language allow all > > > > > > request_header_access Mime-Version allow all > > > > > > request_header_access Retry-After allow all > > > > > > request_header_access Title allow all > > > > > > request_header_access Connection allow all > > > > > > request_header_access Proxy-Connection allow all > > > > > > request_header_access User-Agent allow all > > > > > > request_header_access Cookie allow all > > > > > > request_header_access All deny all > > > > > > > > > > > > > > > > > > # > > > > > > # Add any of your own refresh_pattern entries above these. > > > > > > # > > > > > > #refresh_pattern ^ftp: 1440 20% 10080 > > > > > > #refresh_pattern ^gopher: 1440 0% 1440 > > > > > > #refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 > > > > > > #refresh_pattern (Release|Packages(.gz)*)$ 0 20% > 2880 > > > > > > #refresh_pattern . 0 20% 4320 > > > > > > > > > ################################ > > > > > > acl me proxy_auth ye-1 > > > > > > cache_peer my.proxy.com <http://my.proxy.com> > <http://my.proxy.com <http://my.proxy.com>> > > <http://my.proxy.com/ <http://my.proxy.com/> > <http://my.proxy.com/ <http://my.proxy.com/>>> parent 31280 > > > login=user1:password1 no-query name=a1 > > > > > > cache_peer_access a1 allow me > > > > > > cache_peer_access a1 deny all > > > > > > > > > _______________________________________________ > > > squid-users mailing list > > > squid-users@xxxxxxxxxxxxxxxxxxxxx > <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx> > > <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx > <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx>> > > > http://lists.squid-cache.org/listinfo/squid-users > <http://lists.squid-cache.org/listinfo/squid-users> > > <http://lists.squid-cache.org/listinfo/squid-users > <http://lists.squid-cache.org/listinfo/squid-users>> > > > > > > _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
