Hey Rob, it’s a great question. I will assume you are using squid 5.x since it’s the stable one. There is a configuration reference documentation at: http://www.squid-cache.org/Versions/v5/cfgman/ And the relevant one is ‘acl’: http://www.squid-cache.org/Versions/v5/cfgman/acl.html In the config we the next options: * dstdom_regex * dstdomain * ssl::server_name * ssl::server_name_regex Each one of them is different in two things. The dstdomain is happening when the http request verification is done, usually in the http_access part. So for a CONNECT request in your case it should work but will only work for http_access rules ie allow or deny the connection to being proxied by the proxy. All this happens before the bump stage. The dstdom_regex is doing the same but with another "lookup" mechanism. Just to illustrate we will use the domain www.example.com. With dstdomain .example.com which is a wildcard domain the www.example .com will match since what it does it.. Take apart the .example.com into .com and .example and example (logically, not the code actually does..) Then when a request for www.example-2.com arrives the dstdomain acl checks the strings one by one ie: com example-2 www So the "com" would match and since it has a dot "." as a prefix it continues to the next part of the domain name ie: example-2 and it will try to match it to "example" from the ".example.com'. then it will fail and there for it will declare the specific definition in the acl as "do not match. But for www.example .com it would be different: Squid will find that the .com is in the "com" part of www.example.com and will continue to the next part ie "example" and it will match. Then it will see the dot "." which means that all subdomains are a part of this rule ie both example.com and any other domain under .example.com will match so... www.example.com and www2.example.com and w.example.com will match and this is compared to example.com with dstdomain example.com it will only match a full match of example.com but not www.example.com and it's pointless to define both: - example.com - .example.com In the same acl file. Squid will compare it to the basic proxy url request and the Host: header inside of it. (Amos might remember which one of these wins or which one is the main one) So that's how a dstdomain test works. The dstdom_regex is applied on the same "part" of the proxy request ie request url and Host header but... Tries to match it with a regex that is a much more "CPU" intensive but can do magic if you know how to use it right. The pros are that you can use one regex to match about 100 domains however dstdomain is much faster in many cases. To test regex in general I love: https://rubular.com/ and it's very simple to put a line with the url or the domain itself and write the pattern and then get instant result. There are many sites which does this exact same thing. You need: activate.redshift3d.com for the test. For the server_name dstdom and regex the difference is on what part of the "known" information on the request done. Ie it's not on the known plain text proxy request but a "peek" into the client or server TLS part of the connection bumping. It's possible on TLS 1.2 with ease but 1.3 with encryption it's a whole other story. (Alex and his team is more involved in this part then me) It's a bit more complex in the server_name part since from what I remember the server_name can be the SNI itself or parts of the certificate of the server. In the certificate of the server in many cases there is no real domain but a catch all ie multidomain which is for example: *.example.com The ssl::server_name is like dstdomain so .example.com should match both SNI with www.example.com and the *.example.com If it doesn't then it’s a bug.. About the ssl::server_name_regex you will need to test and match the regex against couple possibilities that are in the certificate ie: *.example.com activation.example.com etc... The current certificate I see is: Certificate: Data: Version: 3 (0x2) Serial Number: c0:71:02:fc:1f:e5:64:5b:0d:d6:ae:c8:1a:17:e6:80 Signature Algorithm: sha256WithRSAEncryption Issuer: C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA Validity Not Before: Dec 14 00:00:00 2022 GMT Not After : Jan 12 23:59:59 2024 GMT Subject: CN = activate.redshift3d.com Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public-Key: (2048 bit) Modulus: 00:cd:f9:bd:58:a4:7a:16:3f:23:12:9d:e0:1e:39: a0:ef:66:ae:b3:ae:32:5c:69:9e:cc:89:83:fb:a2: e0:c7:9d:86:aa:e6:9b:b3:d9:0c:bb:35:36:2f:cf: be:ec:1e:62:ca:d8:ab:16:66:6e:00:8d:f7:42:3d: 5b:d6:bf:a0:30:a1:c2:6f:5e:69:76:ef:0c:38:73: d2:e8:42:b8:83:04:a3:2c:da:22:a4:c4:13:71:38: 36:00:1b:e7:b3:f0:fe:6e:59:17:11:e7:0a:81:3e: 04:b2:ad:e0:61:c1:15:f3:9a:36:33:24:a6:f1:0d: b6:e6:32:91:34:54:7c:da:40:14:72:06:25:91:78: 75:07:84:62:47:7b:3f:05:60:e6:35:d3:27:55:1e: ff:23:82:18:09:f5:8a:0f:a6:cb:02:bc:d6:09:98: 54:5b:10:e3:d2:09:4e:31:2a:75:42:29:00:86:9b: f9:78:2f:fd:99:9e:2e:10:3e:bc:84:f5:9b:3a:3f: fa:d9:b6:20:c0:2e:86:f3:de:14:be:3a:1a:05:30: 3b:00:e6:50:8d:4c:13:a3:97:dc:f4:03:9e:84:31: 0d:23:f2:02:50:6d:53:a5:30:03:b0:fd:f2:46:64: 49:22:a4:d1:5a:00:fb:78:37:fc:51:b1:f0:6e:41: 3b:5d Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Authority Key Identifier: keyid:8D:8C:5E:C4:54:AD:8A:E1:77:E9:9B:F9:9B:05:E1:B8:01:8D:61:E1 X509v3 Subject Key Identifier: 5D:BC:D9:F2:C4:99:83:5E:C0:3A:F1:BF:FE:F2:E3:92:92:F1:F0:29 X509v3 Key Usage: critical Digital Signature, Key Encipherment X509v3 Basic Constraints: critical CA:FALSE X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication X509v3 Certificate Policies: Policy: 1.3.6.1.4.1.6449.1.2.2.7 CPS: https://sectigo.com/CPS Policy: 2.23.140.1.2.1 Authority Information Access: CA Issuers - URI:http://crt.sectigo.com/SectigoRSADomainValidationSecureServerCA.crt OCSP - URI:http://ocsp.sectigo.com X509v3 Subject Alternative Name: DNS:activate.redshift3d.com, DNS:www.activate.redshift3d.com CT Precertificate SCTs: Signed Certificate Timestamp: Version : v1 (0x0) Log ID : 76:FF:88:3F:0A:B6:FB:95:51:C2:61:CC:F5:87:BA:34: B4:A4:CD:BB:29:DC:68:42:0A:9F:E6:67:4C:5A:3A:74 Timestamp : Dec 14 15:34:39.227 2022 GMT Extensions: none Signature : ecdsa-with-SHA256 30:44:02:20:6E:D0:96:68:F3:07:A0:5F:DD:C5:B2:61: 2F:51:6B:06:4A:3C:7F:9B:DA:08:6D:5A:31:0D:B8:0B: 83:7D:D6:37:02:20:64:66:3F:A2:40:87:78:9E:C8:90: 7C:EE:7C:77:3C:BB:43:C1:9F:52:54:F1:04:85:B6:AC: 6B:F2:36:8E:E3:03 Signed Certificate Timestamp: Version : v1 (0x0) Log ID : 3B:53:77:75:3E:2D:B9:80:4E:8B:30:5B:06:FE:40:3B: 67:D8:4F:C3:F4:C7:BD:00:0D:2D:72:6F:E1:FA:D4:17 Timestamp : Dec 14 15:34:39.153 2022 GMT Extensions: none Signature : ecdsa-with-SHA256 30:44:02:20:69:BA:F2:DD:32:B3:48:0C:26:E3:FB:F0: DD:4E:5E:10:95:1E:B4:59:5B:67:64:C4:F2:40:7C:A8: 62:35:71:FE:02:20:13:15:41:26:E5:9C:DB:34:DA:D7: B9:CB:B3:1A:4E:33:C7:46:7F:D9:93:45:25:7F:DE:A3: 72:EB:07:03:DB:C5 Signed Certificate Timestamp: Version : v1 (0x0) Log ID : EE:CD:D0:64:D5:DB:1A:CE:C5:5C:B7:9D:B4:CD:13:A2: 32:87:46:7C:BC:EC:DE:C3:51:48:59:46:71:1F:B5:9B Timestamp : Dec 14 15:34:39.124 2022 GMT Extensions: none Signature : ecdsa-with-SHA256 30:45:02:20:2C:E2:85:9C:A6:54:1B:1C:31:E5:F8:37: E9:CD:09:8B:D8:26:29:E4:C7:65:94:9C:FF:32:D2:41: CD:16:A3:51:02:21:00:A0:2F:C3:F7:A6:55:3B:21:EB: 9B:CA:6E:4E:07:A2:8C:40:4B:E2:27:D6:82:44:0F:09: C9:F7:7D:1B:72:6F:13 Signature Algorithm: sha256WithRSAEncryption 25:bd:bb:de:57:c0:7f:07:5e:18:62:2e:0b:d3:03:54:a7:45: ab:c6:1f:e2:f6:58:ff:6e:8e:6b:4f:09:9a:87:66:32:81:7f: f4:35:4f:7e:65:e5:6a:04:d6:62:62:ff:d9:3a:f2:6f:19:ba: fa:e6:35:0e:2a:44:5c:3b:ee:9d:97:72:05:86:0c:4c:01:c1: f0:8c:21:c1:c4:84:54:d8:a8:05:25:18:72:db:f7:53:9b:f1: 13:d6:0b:bc:92:6e:01:e3:fd:de:a1:45:e9:29:37:e1:2e:64: 36:b4:4d:38:c1:60:02:6a:17:3d:87:a2:5f:33:3b:86:eb:0d: cc:dd:fa:d4:43:58:50:43:e7:b7:ec:0a:4f:86:72:15:e5:30: c9:bb:5f:0b:83:9c:26:6f:60:49:dd:1a:7c:92:45:45:4e:b5: ce:cd:64:8c:12:83:e9:3d:5c:6b:65:97:75:99:4c:66:eb:d0: 3a:ca:18:62:8a:08:07:16:ab:09:66:bd:65:43:94:00:d9:79: 3e:84:b4:60:7d:7e:f9:09:3c:fe:2d:ad:98:94:17:0c:24:8f: e1:a2:74:b6:3b:68:c0:01:f9:67:e8:b9:d2:6a:65:9e:99:a3: 4a:5f:39:31:ae:c1:59:02:7b:ef:db:b2:94:06:f8:1a:74:c1: d7:5b:5b:6a So the DNS names it will check are: Subject: CN = activate.redshift3d.com DNS:activate.redshift3d.com, DNS:www.activate.redshift3d.com So to summarize the checks of ssl::server_name/ will be done on: * activate.redshift3d.com * activate.redshift3d.com * www.activate.redshift3d.com So . redshift3d.com ssl::server_name should match the certificate. If for any reason it doesn't work you can try ssl::server_name_regex with something like: (^|\.)activate\.redshift3d\.com$ Or just to verify if there is a bug in squid code try: (^|\.)activate\.redshift3d\.com Now, the splice should be able to take into account also dstdomain and dstdom_regex but it should match them only if they exist in a plain text form like in any simple forward proxy CONNECT request. If for any reason it doesn’t work we should investigate what might cause this issue. I hope the scroll I wrote make sense to you and with hopes it will clear out the doubts about the wiki article you mentioned. I believe this is considered a summary of the subject and if Alex and others might think so it can be converted into an example article in the wiki. Let me know if this makes sense and resolve the issue. Yours, Eliezer From: robert k Wild <robertkwild@xxxxxxxxx> Sent: Thursday, June 29, 2023 12:18 To: ngtech1ltd@xxxxxxxxx Cc: Squid Users <squid-users@xxxxxxxxxxxxxxxxxxxxx> Subject: Re: make URL bypass squid proxy very clever, so you bunch all the acls up acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom NoSSLInterceptDstDomFile NoSSLInterceptRegEx NoSSLInterceptRegExFile the key word is "any-of" ie if the url hits any one do that first what about instead of making it ssl::server_name_regex i make it dstdom_regex On Thu, 29 Jun 2023 at 01:38, <mailto:ngtech1ltd@xxxxxxxxx> wrote: Hey Rob, The first thing is to allow the domain in the http_acces just to be sure and use a basic deny all bottom line. Let me try to simplify your squid.conf In a link: https://gist.github.com/elico/b49f4a28d4b5db5ba882b10d40872d5e In plain text: ## START OF FILE # SSL Interception basic rules acl DiscoverSNIHost at_step SslBump1 acl NoSSLInterceptRegEx ssl::server_name_regex (^|.*\.)redshift3d\.com$ acl NoSSLInterceptRegExFile ssl::server_name_regex "/usr/local/squid/etc/no-intercept-ssl-regex.txt" acl NoSSLInterceptDstDom ssl::server_name .redshift3d.com acl NoSSLInterceptDstDomFile ssl::server_name "/usr/local/squid/etc/no-intercept-ssl-dstdom.txt" ## Any of will test what ever rule match first in a first match/hit fasion acl NoSSLInterceptAnyOf any-of NoSSLInterceptDstDom NoSSLInterceptDstDomFile NoSSLInterceptRegEx NoSSLInterceptRegExFile ssl_bump peek DiscoverSNIHost ssl_bump splice NoSSLInterceptAnyOf ssl_bump bump all #SSL Bump port http_port 3128 ssl-bump cert=/usr/local/squid/etc/ssl_cert/myCA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=4MB sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB ## http_access acls, will apply on incomming requests and not on responses acl special_url_regex url_regex https?://(^|.*\.)redshift3d\.com\/ acl special_url_regex_file url_regex "/usr/local/squid/etc/special_url_regex.txt" acl special_url_dst_dom dstdomain .redshift3d.com acl special_url_dst_dom_file dstdomain "/usr/local/squid/etc/special_url_dstdom.txt" acl special_url_any_of any-of special_url_dst_dom special_url_dst_dom_file special_url_regex special_url_regex_file acl localnet src http://192.168.0.0/16 acl localnet src http://10.0.0.0/8 http_access allow localnet special_url_any_of http_access deny all ## END OF FILE Once the above will work try to add other http_access rule like reply access rules Let me know what happens, Eliezer From: robert k Wild <mailto:robertkwild@xxxxxxxxx> Sent: Tuesday, June 27, 2023 09:36 To: mailto:ngtech1ltd@xxxxxxxxx Cc: Squid Users <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx> Subject: Re: make URL bypass squid proxy Hi Eliezer, this is a snippet of my whitelist and no intercept SSL config #SSL Interception acl DiscoverSNIHost at_step SslBump1 acl NoSSLIntercept ssl::server_name_regex "/usr/local/squid/etc/interceptssl.txt" ssl_bump peek DiscoverSNIHost ssl_bump splice NoSSLIntercept ssl_bump bump all # #SSL Bump http_port 3128 ssl-bump cert=/usr/local/squid/etc/ssl_cert/myCA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=4MB sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB # #deny up MIME types acl upmime req_mime_type "/usr/local/squid/etc/mimedeny.txt" # #deny URL links acl url_links url_regex "/usr/local/squid/etc/linksurl.txt" # #allow special URL paths acl special_url url_regex "/usr/local/squid/etc/urlspecial.txt" # #deny down MIME types acl downmime rep_mime_type "/usr/local/squid/etc/mimedeny.txt" # http_reply_access allow special_url http_reply_access deny downmime #http_access deny upmime #http_access deny url_links # #HTTP_HTTPS whitelist websites acl whitelist ssl::server_name_regex "/usr/local/squid/etc/urlwhite.txt" # http_access allow activation whitelist http_access deny all so basically no SSL interception #SSL Interception acl DiscoverSNIHost at_step SslBump1 acl NoSSLIntercept ssl::server_name_regex "/usr/local/squid/etc/interceptssl.txt" ssl_bump peek DiscoverSNIHost ssl_bump splice NoSSLIntercept ssl_bump bump all and whitelisting #HTTP_HTTPS whitelist websites acl whitelist ssl::server_name_regex "/usr/local/squid/etc/urlwhite.txt" in both txt files ie /usr/local/squid/etc/interceptssl.txt /usr/local/squid/etc/urlwhite.txt i have a URL that first i have to whitelist and then if i want squid not to inspect the url traffic i put it in the SSL interception (i do this as some websites dont like MITM ) but even putting the URL in question in both files im still having issues with this website ie its still being detected that its passing through a proxy thanks, rob On Mon, 26 Jun 2023 at 23:35, <mailto:mailto:ngtech1ltd@xxxxxxxxx> wrote: Hey Robert, I am not sure what forward proxy setup you have there. A simple forward proxy? What tool are you using for whitelisting? You can use an external acl helper to allow dynamic updates of the whitelists or to periodic update your lists and reload. It will depend on the size of your lists. What OS are you using for your squid proxy? More details will help us help you. Eliezer From: squid-users <mailto:mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of robert k Wild Sent: Monday, June 26, 2023 22:25 To: Squid Users <mailto:mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx> Subject: make URL bypass squid proxy hi all, i have set up squid for url whitelisting and no intercept SSL (see below) https://wiki.squid-cache.org/ConfigExamples/Caching/AdobeProducts but some websites i want the client to bypass the squid proxy and go straight to the website as i think this is why a url isnt working even when i add the url to both files ie urlwhite and no intercept SSL thanks, rob -- Regards, Robert K Wild. -- Regards, Robert K Wild. -- Regards, Robert K Wild. _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
