I'm sorry. I forgot to attach the configuration file used in the verification (partially masked). 2024年9月5日(木) 17:58 にば <taku0919.taku@xxxxxxxxx>: > > Hi Alex > > Thanks for the reply ! > > I took the advice I received, reviewed the verification details, and > verified again with the two recommended steps. > The new verification includes the following four patterns: > 1. successful communication of a valid request to an allowed site > [command] > curl https://pypi.org/ -v --cacert squid.crt -k > > 2. communication of invalid requests to allowed sites is denied > [command] ※Invalid header host > curl https://pypi.org/ -H "Host: www.yahoo.co.jp" -v --cacert squid.crt -k > > 3. communication of valid requests to prohibited sites is denied > [command] > curl https://www.yahoo.co.jp/ -v --cacert squid.crt -k > > 4. communication of invalid requests to prohibited sites is denied > [command] > curl https://www.yahoo.co.jp/ -H "Host:" -v --cacert squid.crt -k > > > STEP１ > ＞1. Remove all of your current http_access rules. Keep ACLs. Perform > ＞host_verify_strict and access tests to confirm that all valid requests > ＞are denied and all invalid requests are rejected. If necessary, ask > ＞questions, file bug reports, patch Squid, and/or adjust your > ＞configuration to pass this test. > > ・Results for validation patterns > １．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0 > ２．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0 > ３．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0 > ４．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0 > > STEP２ > ＞2. Copy http_access rules, with comments, from generated > ＞squid.conf.default. Insert your own access rules in the location marked > ＞by "INSERT YOUR OWN RULE(S) HERE" comment. Perform host_verify_strict > ＞and access tests to confirm that all valid requests to banned sites are > ＞denied, all other valid requests are allowed, and all invalid requests > ＞are rejected. If necessary, ask questions, file bug reports, patch > ＞Squid, and/or adjust your configuration to pass this test. > > ・Results for validation patterns > １．200 OK > ２．409 Conflict、X-Squid-Error: ERR_CONFLICT_HOST 0 > ３．409 Conflict、X-Squid-Error: ERR_CONFLICT_HOST 0 > ４．200 OK > > So my question is, in STEP 1. > ・I expected to get a different response for a valid request than for > an invalid request, is this result correct? > In STEP 2. > ・4. still returns 200 OK. Is this due to an error in the existing > configuration? Or do I need to add a new setting? > > I am not a good English, so I may not have understood Alex's advice exactly. > If there are still mistakes, please point them out to me. > > Thank you in advance. > > 2024年8月30日(金) 22:27 Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>: > > > > On 2024-08-29 22:28, にば wrote: > > > > > With the newly reviewed configuration in the attachment > > > > OT: Please note that your configuration does not follow the recommended > > http_access rules order template in squid.conf.default and might, > > depending on your deployment environment, allow Squid to be used for > > attacks on 3rd party resources (e.g., ssh services). This observation is > > not related to your primary question and your "ban certain sites" goal. > > Following suggestions at the end of this email should fix this problem. > > > > > > > I found the following statement in the following official document > > > https://www.squid-cache.org/Doc/config/host_verify_strict/ > > > > > > ＞ * The host names (domain or IP) must be identical, > > > ＞ but valueless or missing Host header disables all checks. > > > > > > So I ran an additional validation with an empty Host value, and the > > > request succeeded for a domain that was not in the whitelist. > > > The curl command for verification is below, and as before, only > > > .pypi.org is allowed in the whitelist. > > > > > > date;curl https://www.yahoo.co.jp/ -H "Host:" -v --cacert squid.crt -k > > > > > > Is it possible for Squid to prevent such requests as well? > > > > Yes, a req_header ACL should be able to detect such requests (i.e. > > requests without a Host header or with an empty Host header value). > > However, I suspect that "missing Host" is _not_ the problem you should > > be solving (as detailed below). > > > > > > > I was able to confirm that any one of the SNI, IP, or Host in the > > > request is incorrect (not whitelist allowed) > > > and Squid will correctly check and return a 409 Conflict. > > > > IMHO, you should target/validate a different set of goals/conditions: > > > > * A valid request targeting a banned site should be denied (HTTP 403 > > response, %Ss=TCP_DENIED, %err_code=ERR_ACCESS_DENIED). This denial > > should be triggered by an "http_access deny" rule, preferably an > > explicit one. This denial will _not_ happen (and the request will > > instead be forwarded to the banned site it targets) if you replace all > > your http_access rules with a single "http_access allow all" line. This > > denial does not depend on host_verify_strict and underlying code. > > > > * An invalid request should be rejected (HTTP 4xx response). This > > includes, but is not limited to, host_verify_strict-driven rejections. > > This rejection should happen even if you replace all your http_access > > rules with a single "http_access allow all" line. > > > > AFAICT, your current configuration does not reach "deny valid requests > > targeting banned sites" goal while your question implies that you are > > incorrectly relying on host_verify_strict to perform that denial. > > > > > > I recommend the following: > > > > 1. Remove all of your current http_access rules. Keep ACLs. Perform > > host_verify_strict and access tests to confirm that all valid requests > > are denied and all invalid requests are rejected. If necessary, ask > > questions, file bug reports, patch Squid, and/or adjust your > > configuration to pass this test. > > > > 2. Copy http_access rules, with comments, from generated > > squid.conf.default. Insert your own access rules in the location marked > > by "INSERT YOUR OWN RULE(S) HERE" comment. Perform host_verify_strict > > and access tests to confirm that all valid requests to banned sites are > > denied, all other valid requests are allowed, and all invalid requests > > are rejected. If necessary, ask questions, file bug reports, patch > > Squid, and/or adjust your configuration to pass this test. > > > > > > HTH, > > > > Alex. > > > > > > > 2024年8月8日(木) 21:33 Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>: > > >> > > >> On 2024-08-06 20:59, にば wrote: > > >> > > >>> When using Squid transparently, is it possible to control the > > >>> whitelist of the domain to connect to and inspect the Host field in > > >>> the request header together? > > >> > > >> Short answer: Yes. > > >> > > >> > > >>> According to the verification results, the Host field can be inspected > > >>> by "host_verify_strict on" in squid-transparent.conf, but it seems > > >>> that the whitelist is not controlled. > > >> > > >> AFAICT, the configuration you have shared allows all banned[1] traffic > > >> to/through https_port. For the problematic test case #5: > > >> > > >> All these http_access rules do _not_ match: > > >> > > >>> http_access allow localnet whitelist > > >>> http_access deny localnet whitelist_https !https_port > > >>> http_access deny localnet whitelist_transparent_https !https_port > > >> > > >> > > >> And then this next rule matches and allows traffic through: > > >> > > >>> http_access allow https_port > > >> > > >> > > >> This last http_access rule is not reached: > > >> > > >>> http_access deny all > > >> > > >> > > >> N.B. The above analysis assumes that your https_port ACL is explicitly > > >> defined in your squid.conf to match all traffic received at https_port. > > >> If you do not have such an ACL defined, then you need to fix that > > >> problem as well. I recommend naming ACLs differently from directive > > >> names (e.g., "toHttpsPort" rather than "https_port"). > > >> > > >> > > >> Please note that Squid v4 is not supported by the Squid Project and is > > >> very buggy. I recommend using Squid v6 or later. > > >> > > >> > > >> HTH, > > >> > > >> Alex. > > >> [1] Here, "banned" means "_not_ matching whitelist ACL". > > >> > > >> > > >>> ■Configuration Details > > >>> 〇squid-transparent.conf（Excerpts） > > >>> #Whitelist > > >>> acl whitelist dstdomain "/etc/squid/whitelist" > > >>> acl whitelist dstdomain "/etc/squid/whitelist_transparent" > > >>> acl whitelist_https dstdomain "/etc/squid/whitelist_https" > > >>> acl whitelist_transparent_https dstdomain > > >>> "/etc/squid/whitelist_transparent_https" > > >>> > > >>> proxy_protocol_access allow localnet > > >>> proxy_protocol_access deny all > > >>> http_access allow localnet whitelist > > >>> http_access deny localnet whitelist_https !https_port > > >>> http_access deny localnet whitelist_transparent_https !https_port > > >>> > > >>> # Handling HTTP requests > > >>> http_port 3129 intercept > > >>> # Handling HTTPS requests > > >>> https_port 3130 intercept tcpkeepalive=60,30,3 ssl-bump > > >>> generate-host-certificates=on dynamic_cert_mem_cache_size=20MB > > >>> tls-cert=/etc/squid/ssl/squid.crt tls-key=/etc/squid/ssl/squid.key > > >>> cipher=HIGH:MEDIUM:!LOW:!RC4:!SEED:!IDEA:!3DES:!MD5:!EXP:!PSK:!DSS > > >>> options=NO_TLSv1,NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE > > >>> tls-dh=prime256v1:/etc/squid/ssl/bump_dhparam.pem > > >>> # Start up for squid process > > >>> http_port 3131 > > >>> http_access allow https_port > > >>> acl allowed_https_sites ssl::server_name "/etc/squid/whitelist" > > >>> acl allowed_https_sites ssl::server_name "/etc/squid/whitelist_transparent" > > >>> acl allowed_https_sites ssl::server_name "/etc/squid/whitelist_https" > > >>> acl allowed_https_sites ssl::server_name > > >>> "/etc/squid/whitelist_transparent_https" > > >>> > > >>> http_access deny all > > >>> > > >>> # strict setting > > >>> host_verify_strict on > > >>> > > >>> # SSL_BUMP > > >>> sslcrtd_program /usr/lib64/squid/security_file_certgen -s > > >>> /var/lib/squid/ssl_db -M 20MB > > >>> acl step1 at_step SslBump1 > > >>> acl step2 at_step SslBump2 > > >>> acl step3 at_step SslBump3 > > >>> > > >>> ssl_bump bump all > > >> > > >> > > >>> ■Verification of Settings > > >>> I ran the curl command from each of the client environments that use Squid. > > >>> 1. if SNI, Destination IP, and HeaderHost are correct, the user should > > >>> be able to connect to pypi.org > > >>> Command: > > >>> date;curl https://pypi.org/ -v --cacert squid_2.crt -k > > >>> Result: OK > > >>> > > >>> 2. rejection of communication to pypi.org if SNI is correct but > > >>> destination IP and HeaderHost are incorrect > > >>> Command: > > >>> date;curl https://pypi.org/ --resolve pypi.org:443:182.22.24.252 -H > > >>> "Host: www.yahoo.co.jp" -v --cacert squid_2.crt -k > > >>> Result: OK (409 Conflict is returned) > > >>> > > >>> 3. rejection of communication to pypi.org if SNI and destination IP > > >>> are correct and HeaderHost is incorrect > > >>> Command: > > >>> date;curl https://pypi.org/ -H "Host: www.yahoo.co.jp" -v --cacert > > >>> squid_2.crt -k > > >>> Result: OK (409 Confilic returned) > > >>> > > >>> 4. rejection of communication to pypi.org if SNI and HeaderHost are > > >>> correct but destination IP is incorrect > > >>> Command: > > >>> date;curl https://pypi.org/ --resolve pypi.org:443:182.22.24.252 -v > > >>> --cacert squid_2.crt -k > > >>> Result: OK (409 Confilic returned) > > >>> > > >>> 5. if SNI, destination IP, and HeaderHost are all invalid (yahoo.co.jp > > >>> not registered in whitelist), communication will be rejected > > >>> Command: > > >>> date;curl https://yahoo.co.jp/ -v --cacert squid_2.crt -k > > >>> Result: NG (301 Moved Permanently is returned, but it appears that the > > >>> communication is reaching yahoo.co.jp) > > >> > > >> > > >> > > >> > > >> _______________________________________________ > > >> squid-users mailing list > > >> squid-users@xxxxxxxxxxxxxxxxxxxxx > > >> https://lists.squid-cache.org/listinfo/squid-users > >
Attachment:
step2.squid-transparent.conf
Description: Binary data
Attachment:
step1.squid-transparent.conf
Description: Binary data
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx https://lists.squid-cache.org/listinfo/squid-users
