Hello Alex, Thank you for clarifying, that makes a lot more sense when you put it like that. So the request to go to that internal website is not being authenticated when browsing to it via hostname. But it *does* authenticate when browsing to it manually via IP. So one of the ACL's we have in place is probably trying to pass this traffic as unauthenticated, and this is causing the request to fail. Prior to the Squid upgrade, when users browed to these sites, they needed to enter their AD credentials because the proxy didn't automatically let them in. We're still trying to figure out which one did that. The goal: Browse internal destinations on other subnets by hostname without being blocked. FWIW, the error to each internal site is a little bit different. Hexcelssp, 10.96.15.174. gives me a "We're having trouble finding that site". Hexnt2067, 10.96.102.67, gives me a proper proxy blocking error. "The requested URL could not be retrieved". -----Original Message----- From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of Alex Rousskov Sent: Wednesday, September 4, 2024 3:16 PM To: squid-users@xxxxxxxxxxxxxxxxxxxxx Subject: Re: Unable to access internal resources via hostname Caution: This email originated from outside of Hexcel. Do not click links or open attachments unless you recognize the sender and know the content is safe. On 2024-09-04 12:25, Piana, Josh wrote: > That's REALLY strange that its getting blocked by "http_access deny > block_user". That's an ACL that is supposed to just deny "known good" > users that we don't want to have access to the user proxy. This ACL > references the "block_user" list which includes guest accounts, vender > accounts, and a few users. The account I'm testing with, "jpiana" is > not on that list. I wonder why I'm getting filtered here and not the > very bottom of the squid.conf where we have the "http_access deny all" > ACL. I know a lot of things have changed since you wrote the above, but I wanted to answer that still-relevant question: The test transaction gets an HTTP 407 response not because its client name matches a block_user ACL entry. The test transaction gets an HTTP 407 response because the user has not authenticated itself -- Squid sends HTTP 407 response to request authentication. If you are still having problems after changing the test proxy and its configuration (as you discussed in your recent posts), please restate the primary problem and share debugging log of a test transaction again. Cheers, Alex. > Maybe the order of my ACL's is wrong? > > In regards to the "local_dst_dom", I've setup a a list here under this path: "/etc/squid/local_dst_dom", to immitate the other ACL rules. When I added the ".hexcelssp.com" address to this list, it failed to work. Most likely because this isn't a domain. > > FWIW, I just want ad.arc-tech.com devices to be allowed to reach all other ad.arc-tech.com devices via hostname. I'm willing to remove any other ACL that's getting in the way of that in order to make this work. This whole config that we have has been pieced together and I'd like to get it to the way it's supposed to be. > > What do you recommend? I can send the whole config again, exactly as we have it now, and see what we can fix/remove/replace. > > Appreicate you helping, > Josh > > -----Original Message----- > From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On > Behalf Of Alex Rousskov > Sent: Tuesday, September 3, 2024 4:43 PM > To: squid-users@xxxxxxxxxxxxxxxxxxxxx > Subject: Re: Unable to access internal resources via > hostname > > Caution: This email originated from outside of Hexcel. Do not click links or open attachments unless you recognize the sender and know the content is safe. > > > Hi Josh, > > Thank you for sharing debugging logs. "CONNECT hexcelssp" > transaction is being denied by the following rule: > > http_access deny block_user > > None of the ~10 http_access rules above the quoted one matched. Which of those rules do you expect to match that "CONNECT hexcelssp" request (_before_ the rule quoted above denies the request and asks the client to authenticate)? I suspect it is the local_dst_dom rule. > > local_dst_dom ACL will not match this transaction. You have changed the definition of that ACL from a hard-coded (in squid.conf) "arcgate" name to an unknown to me set of names inside "etc/squid/local_dst_dom" file. > I wonder whether that file path is correct: Did you mean that path to be relative (i.e. "etc...") rather than absolute (i.e. "/etc...")? Try using an absolute path and double check that the file contains "hexcelssp" name. > > > HTH, > > Alex. > > > 2024/09/03 13:14:11.467 kid1| 85,2| client_side_request.cc(758) > clientAccessCheckDone: The request CONNECT hexcelssp:443 is > AUTH_REQUIRED; last ACL checked: block_user > > > > > On 2024-08-29 10:34, Piana, Josh wrote: >> Good morning Alex, >> >> I've added the following to my squid.conf file # logformat custom >> %err_code/%err_detail # access_log daemon:/var/log/squid/access,log >> custom >> >> I've also enabled debugging options: >> # debug_options ALL,9 >> >> I've also cleaned up our ACL's to better reflect what is going on: >> # acl src_self src 127.0.0.0/8 >> # acl src_self src 10.46.11.69 >> # acl dst_self dst 127.0.0.0/8 >> # acl dst_self dst 10.46.11.69 >> >> # acl from_arc src 10.46.0.0/15 >> >> # acl local_dst_addr dst 10.0.0.0/8 >> # acl local_dst_addr dst 172.0.0.0/8 >> >> When accessing internal devices via IP, it works. >> I think this is because of this ACL: >> # http_access allow local_dst_addr >> # http_reply_access allow local_dst_addr >> >> But when I access those same internal devices via hostname, it fails. >> I'm using this ACL, which I just created a separate file for: >> acl local_dst_dom dstdomain "etc/squid/local_dst_dom" >> http_access allow local_dst_dom >> http_reply_access allow local_dst_dom >> >> I added an ACL file because I figure it will be more extensive than the more simple IP range ACL's. >> I've added a few local websites to the "local_dst_dom" list, but I'm unable to get this to resolve without using IP. >> >> For instance: "https://hexcelssp/"; is the name of one of our internal websites for our company. >> This doesn't load under our current config but "https://10.96.14.174/"; DOES load. >> >> I've added "hexnt2067" as well, which you can see me testing in the below logs. >> Again, this DOES work with its IP, 10.96.102.67. >> >> I'm not getting a proxy error message when I try to browse to that URL though, we just can't reach it. >> >> Here's the log results after enabling debugging and better log features: >> >> 29/Aug/2024:10:25:33 -0400.380 10.46.49.190 TCP_DENIED/407 4132 >> CONNECT hexcelssp:443 - HIER_NONE/- text/html >> 29/Aug/2024:10:25:33 -0400.515 10.46.49.190 NONE_NONE/500 0 CONNECT >> hexcelssp:443 JPIANA@AD.<DOMAIN>.COM HIER_NONE/- - >> 29/Aug/2024:10:25:39 -0400.799 10.46.49.190 TCP_TUNNEL/200 11716 >> CONNECT contile.services.mozilla.com:443 JPIANA@AD.<DOMAIN>.COM >> HIER_DIRECT/34.117.188.166 - >> 29/Aug/2024:10:26:05 -0400.507 10.46.49.190 TCP_DENIED/407 4474 GET >> http://hexnt2067/sites/it/help/SitePages/Home.aspx - HIER_NONE/- >> text/html >> 29/Aug/2024:10:26:05 -0400.646 10.46.49.190 TCP_MISS/500 8320 GET >> http://hexnt2067/sites/it/help/SitePages/Home.aspx >> JPIANA@AD.<DOMAIN>.COM HIER_NONE/- text/html >> 29/Aug/2024:10:26:05 -0400.735 10.46.49.190 TCP_DENIED/403 4440 GET >> http://arcga/ >> te2.ad%2F&data=05%7C02%7Cjosh.piana%40hexcel.com%7C09078e3264324c317c >> 6 >> e08dccc58fcfe%7C4248050df19546d5ac9c0c7c52b04cae%7C0%7C0%7C6386099298 >> 1 >> 3620083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL >> C >> JBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ApuKveTxEh4lWJiyQw3D5 >> A >> Uh9PWut1%2FyxniOr6l1RL4%3D&reserved=0.<domain>.com:8080/squid-interna >> l -static/icons/SN.png - HIER_NONE/- text/html >> 29/Aug/2024:10:26:05 -0400.779 10.46.49.190 TCP_MISS_ABORTED/500 0 >> GET http://hexnt2067/favicon.ico JPIANA@AD.<DOMAIN>.COM HIER_NONE/- >> text/html >> 29/Aug/2024:10:27:13 -0400.036 10.46.49.190 TCP_DENIED/407 4474 GET >> http://hexnt2067/sites/it/help/SitePages/Home.aspx - HIER_NONE/- >> text/html >> 29/Aug/2024:10:27:13 -0400.227 10.46.49.190 TCP_MISS/500 8334 GET >> http://hexnt2067/sites/it/help/SitePages/Home.aspx >> JPIANA@AD.<DOMAIN>.COM HIER_NONE/- text/html >> 29/Aug/2024:10:27:13 -0400.302 10.46.49.190 TCP_DENIED/403 4440 GET >> http://arcga/ >> te2.ad%2F&data=05%7C02%7Cjosh.piana%40hexcel.com%7C09078e3264324c317c >> 6 >> e08dccc58fcfe%7C4248050df19546d5ac9c0c7c52b04cae%7C0%7C0%7C6386099298 >> 1 >> 3620083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL >> C >> JBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ApuKveTxEh4lWJiyQw3D5 >> A >> Uh9PWut1%2FyxniOr6l1RL4%3D&reserved=0.<domain>.com:8080/squid-interna >> l -static/icons/SN.png - HIER_NONE/- text/html >> 29/Aug/2024:10:27:17 -0400.376 10.46.49.190 TCP_DENIED/407 4132 >> CONNECT hexcelssp:443 - HIER_NONE/- text/html >> 29/Aug/2024:10:27:17 -0400.514 10.46.49.190 NONE_NONE/500 0 CONNECT >> hexcelssp:443 JPIANA@AD.<DOMAIN>.COM HIER_NONE/- - >> >> I'm not sure the debugging and extra log details were added correctly, because these look the same. >> >> Thanks, >> Josh >> >> >> -----Original Message----- >> From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On >> Behalf Of Alex Rousskov >> Sent: Wednesday, August 28, 2024 4:01 PM >> To: squid-users@xxxxxxxxxxxxxxxxxxxxx >> Subject: Re: Unable to access internal resources via >> hostname >> >> Caution: This email originated from outside of Hexcel. Do not click links or open attachments unless you recognize the sender and know the content is safe. >> >> >> On 2024-08-28 15:25, Piana, Josh wrote: >> >>> I now understand that Squid 5.5 ... is actually out of date as well. >>> I regret installing and working off of this because I jused assumed >>> the "yum" install command for the Squid distro would give me the >>> most recent, known-good, software. That's completely on me. >> >> Whether yum install installs something production-worthy today is between you and your Linux distro. Squid Project volunteers, including me, do no control what distribution you pick and what that distribution ships. The reason I mentioned v5 support status is to avoid creating a false impression that there are no _other_ (i.e. unreported by you on this thread) problems you may want to know about and to lower your expectations from the level of support you may get from Squid Project volunteers like me. >> >> Said that, I doubt the problem you are trying to solve is v5-specific. >> You may get better diagnostics if you run v6+, and you may receive fewer unusable (with v5) instructions if you run v6+, but the problem is likely to be present in a v6-based setup as well. >> >> >>> From how I understand it, his intentions for internal users >>> accessing internal resources, would skip authentication altogether. >> >> I assume that the problematic test that you want to fix is done using >> "internal users". You have already stated that the test is accessing >> "internal resources". Yet, we see that Squid asks the test client to >> authenticate (TCP_DENIED/407). Thus, >> >> * either local_dst_dom, local_dst_addr, and authless_src ACLs do not >> match test client transactions >> >> * or there is an authentication-triggering directive (e.g., "http_access deny block_user") located above a matching "http_access allow" rule that uses one of those ACLs three. >> >> Ideally, we should figure out which http_access rule matches the problematic transaction and triage from there. Unfortunately, Squid does not make that basic task easy. The best option I can think of is for you to share (privately if needed to avoid publishing sensitive info) a pointer to compressed cache.log file collected while reproducing the problem using a single test transaction after setting debug_options to ALL,9. Squid FAQ has a few hints about collecting relevant info: >> https://wiki/ >> .squid-cache.org%2FSquidFaq%2FBugReporting%23debugging-a-single-trans >> a >> ction&data=05%7C02%7Cjosh.piana%40hexcel.com%7C09078e3264324c317c6e08 >> d >> ccc58fcfe%7C4248050df19546d5ac9c0c7c52b04cae%7C0%7C0%7C63860992981377 >> 6 >> 344%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT >> i >> I6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=D%2BSI8apBgbZJnJfqr0J5WMi >> P >> GtnmIVdrFDPfD1vL6mM%3D&reserved=0 >> >> This "unexpected authentication" mystery may not be directly related to the "HTTP 500 error responses" mystery we discussed earlier, but it may help to fix authentication first. >> >> >> HTH, >> >> Alex. >> >> >>> Here's a small piece, with his notes included, of why I think that. >>> >>> -------------------------------------------------------------------- >>> - >>> - >>> -------------------------------------------------- >>> # >>> -------------------------------------------------------------------- >>> - >>> - >>> ------ >>> # allow without authentication >>> # these rules allow certain connects without user authentication # >>> these allow any protocol/method/etc >>> >>> # ***** IMPORTANT ***** >>> # Adding to these lists also exempts from all content filtering. >>> # In particular, executables will be allowed to download! >>> # ***** IMPORTANT ***** >>> >>> # allow connects to local destinations without authentication # by >>> domain name from URL >>> http_access allow local_dst_dom >>> http_reply_access allow local_dst_dom >>> >>> # by IP address name resolves to >>> http_access allow local_dst_addr >>> http_reply_access allow local_dst_addr >>> -------------------------------------------------------------------- >>> - >>> - >>> -------------------------------------------------- >>> >>> The "http_access allow local_dst_dom" and the following reply statement seems to point toward his intentions when writing this. >>> >>> The only thing in the old config this was referencing was this, and to me, it doesn't make sense: >>> # acl local_dst_dom dstdomain arcgate >>> >>> It's worth noting that despite how awkward this seems, it DOES work. On the old web proxy we're currently still running live. >>> >>> I've gone as far as swapping out just the authentication settings and keeping the entire old config, but it still doesn't work. >>> I've setup a realmd/sssd/kerberos backend instead of Samba/Windbind that the old server uses. >>> >>> To state the issue we're having again - internal resources connecting to other internal resources fails via hostname, but works when using IP. I was able to troubleshoot and conclude DNS was not the issue, as all other aspects are working as intended and they certainly do use DNS. Also pings resolve using hostname from the new squidbox so that implies that DNS is working as expected. >>> >>> Ultimately, here's what I think we want, please feel free to poke holes in it: >>> Allow clients who are accessing other internal rersources to be able to browse to it using its hostname, without needing authentication first. >>> >>> >>> I do intend on adding those parameters you mentioned to the logs so I can get better details within. I also need to look into how to use the debugging features, I'm not familiar with it at all. >>> >>> I apologize for the wall of text, looking forward to what you guys have to say about this. >>> >>> Thanks, >>> Josh >>> >>> -----Original Message----- >>> From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On >>> Behalf Of Alex Rousskov >>> Sent: Wednesday, August 28, 2024 2:31 PM >>> To: squid-users@xxxxxxxxxxxxxxxxxxxxx >>> Subject: Re: Unable to access internal resources via >>> hostname >>> >>> Caution: This email originated from outside of Hexcel. Do not click links or open attachments unless you recognize the sender and know the content is safe. >>> >>> >>> On 2024-08-28 14:18, Alex Rousskov wrote: >>>> On 2024-08-28 11:24, Piana, Josh wrote: >>>> >>>>> Here's the log and (I think) relevant ACL's? >>>> >>>> According to your access.log, Squid denies problematic CONNECT >>>> requests with HTTP 407 errors responses. Usually, that means those >>>> requests match an "http_access deny" rule. Clearly, you expect an >>>> "allow" outcome instead, but it is difficult (for me) to figure out >>>> where your expectations mismatch reality; there are no rules that >>>> explicitly mention hexcelssp domain, for example: Which >>>> "http_access allow" rule do you expect those denied requests to match? >>> >>> Sorry, I probably misinterpreted those access.log records: It looks like the denied (TCP_DENIED/407) access is something you actually expect because you want that test request to be authenticated. The client supplies the necessary credentials in the second request, and then that second request fails with a (rather generic) HTTP 500 error code, without contacting the origin server. >>> >>> I am guessing that you are concerned about that second request/transaction rather than the first one. >>> >>> Squid generates HTTP 500 errors for a variety of different reasons. Are there any messages in cache.log (at default debugging level) that correspond to these failing test transactions? If there are none, please add %err_code/%err_detail to your access_log logformat so that Squid logs more information about the problem to access.log (see logformat and access_log directives in squid.conf.documented for details). >>> >>> >>> Thank you, >>> >>> Alex. >>> >>> >>> >>>> Also, does mgr:ipcache cache manager query confirm that Squid has >>>> read your /etc/hosts file and cached the record you expect it to use? >>>> >>>> Alex. >>>> >>>> >>>>> ------------------------------------------------------------------ >>>>> - >>>>> - >>>>> - >>>>> -------------------------------------- >>>>> # /var/log/squid/access.log results for internal conflicts >>>>> >>>>> 28/Aug/2024:10:57:17 -0400.234 10.46.49.190 TCP_DENIED/407 4132 >>>>> CONNECT hexcelssp:443 - HIER_NONE/- text/html >>>>> 28/Aug/2024:10:57:17 -0400.253 10.46.49.190 NONE_NONE/500 0 >>>>> CONNECT >>>>> hexcelssp:443 JPIANA@AD.<DOMAIN>.COM HIER_NONE/- - >>>>> 28/Aug/2024:10:57:17 -0400.380 10.46.49.190 TCP_DENIED/407 4132 >>>>> CONNECT hexcelssp:443 - HIER_NONE/- text/html >>>>> 28/Aug/2024:10:57:17 -0400.399 10.46.49.190 NONE_NONE/500 0 >>>>> CONNECT >>>>> hexcelssp:443 JPIANA@AD.<DOMAIN>.COM HIER_NONE/- - >>>>> ------------------------------------------------------------------ >>>>> - >>>>> - >>>>> - >>>>> -------------------------------------- >>>>> >>>>> # acl all src all >>>>> >>>>> acl src_self src 127.0.0.0/8 >>>>> acl src_self src 10.46.11.69 >>>>> >>>>> acl dst_self dst 127.0.0.0/8 >>>>> acl dst_self dst 10.46.11.69 >>>>> >>>>> acl from_arc src 10.46.0.0/15 >>>>> >>>>> acl local_dst_addr dst 10.0.0.0/8 >>>>> acl local_dst_addr dst 172.0.0.0/8 acl local_dst_addr dst >>>>> bldg3.<domain>.com acl local_dst_addr dst bldg5.<domain>.com >>>>> >>>>> # these keep URLs of popular local servers from being forwarded >>>>> acl local_dst_dom dstdomain arcgate >>>>> >>>>> # allow connects to local destinations without authentication # by >>>>> domain name from URL >>>>> http_access allow local_dst_dom >>>>> http_reply_access allow local_dst_dom >>>>> >>>>> # by IP address name resolves to >>>>> http_access allow local_dst_addr >>>>> http_reply_access allow local_dst_addr >>>>> >>>>> # allow trusted hosts without authentication # these are just ip's >>>>> on the 10.46.11.x network acl authless_src src "/etc/squid/authless_src" >>>>> http_access allow authless_src >>>>> http_reply_access allow authless_src >>>>> ------------------------------------------------------------------ >>>>> - >>>>> - >>>>> - >>>>> -------------------------------------- >>>>> >>>>> -----Original Message----- >>>>> From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On >>>>> Behalf Of Matus UHLAR - fantomas >>>>> Sent: Wednesday, August 28, 2024 10:47 AM >>>>> To: squid-users@xxxxxxxxxxxxxxxxxxxxx >>>>> Subject: Re: Unable to access internal resources via >>>>> hostname >>>>> >>>>> Caution: This email originated from outside of Hexcel. Do not >>>>> click links or open attachments unless you recognize the sender >>>>> and know the content is safe. >>>>> >>>>> >>>>> On 28.08.24 14:20, Piana, Josh wrote: >>>>>> Hello Squid Support, >>>>> >>>>> This squid user forum FYI >>>>> >>>>>> We are unable to get to internal resources via hostname but using >>>>>> the IP address works fine. Immediately, I thought this was DNS >>>>>> but when I checked the /etc/resolv.conf/ file it was pointing >>>>>> correctly to our Windows DNS server and we can ping all devices >>>>>> using their hostname, just not when browsing to it. This leads >>>>>> me to believe something may be wrong with our squid config. >>>>> >>>>> hard to guess without seeing logs or ACL's. >>>>> >>>>> >>>>> -- >>>>> Matus UHLAR - fantomas, uhlar@xxxxxxxxxxx _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx https://lists.squid-cache.org/listinfo/squid-users _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx https://lists.squid-cache.org/listinfo/squid-users
