Ok, sorry for so many messages. This is the last one :)
In the end what helped was this:
acl internal_digest urlpath_regex +i ^/squid-internal-periodic/store_digest$
always_direct allow internal_digest
never_direct deny internal_digest
So Amos' original idea with ACL was correct, I just had to adjust it a bit.
Looks like "never_direct allow all" which I have later in config affects store_digest requests. Not sure if it's a bug or feature.
Thank you for helping again.
On Thu, Dec 29, 2016 at 4:15 PM, Ivan Larionov <xeron.oskom@xxxxxxxxx> wrote:
Here are some debug logs from FwdState which handles digest request.172.22.13.210 – original squid172.22.8.145 – sibling squid127.0.0.1:18070 – parentAs you can see it uses connection to parent for this request (reusing pconn local=127.0.0.1:44120 remote=127.0.0.1:18070 FD 16 flags=1) which is probably a bug.2016/12/29 15:57:41.121| 17,3| FwdState.cc(332) Start: 'http://172.22.8.145:3128/squid-internal-periodic/store_ 'digest 2016/12/29 15:57:41.121| 17,2| FwdState.cc(133) FwdState: Forwarding client request , url="" href="http://172.22.8.145:3128/squid-internal-periodic/store_digest" target="_blank">http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 15:57:41.121| 17,3| FwdState.cc(387) startConnectionOrFail: http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 15:57:41.121| 17,3| FwdState.cc(806) connectStart: fwdConnectStart: http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 15:57:41.121| 17,3| FwdState.cc(875) connectStart: reusing pconn local=127.0.0.1:44120 remote=127.0.0.1:18070 FD 16 flags=12016/12/29 15:57:41.121| 17,3| FwdState.cc(908) dispatch: : Fetching GET http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 15:57:41.124| 17,3| FwdState.cc(447) unregister: http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 15:57:41.124| 17,2| FwdState.cc(655) handleUnregisteredServerEnd: self=0x1450738*2 err=0 http://172.22.8.145:3128/squid-internal-periodic/store_ digest And peer_select logs:2016/12/29 16:12:41.843| 44,3| peer_select.cc(137) peerSelect: e:=IWV/0x148bae0*2 http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 16:12:41.843| 44,3| peer_select.cc(441) peerSelectFoo: GET 172.22.8.1452016/12/29 16:12:41.843| 44,3| peer_select.cc(446) peerSelectFoo: peerSelectFoo: direct = DIRECT_UNKNOWN (always_direct to be checked)2016/12/29 16:12:41.844| 44,3| peer_select.cc(194) peerCheckAlwaysDirectDone: peerCheckAlwaysDirectDone: DENIED2016/12/29 16:12:41.844| 44,3| peer_select.cc(441) peerSelectFoo: GET 172.22.8.1452016/12/29 16:12:41.844| 44,3| peer_select.cc(454) peerSelectFoo: peerSelectFoo: direct = DIRECT_UNKNOWN (never_direct to be checked)2016/12/29 16:12:41.844| 44,3| peer_select.cc(171) peerCheckNeverDirectDone: peerCheckNeverDirectDone: ALLOWED2016/12/29 16:12:41.844| 44,3| peer_select.cc(177) peerCheckNeverDirectDone: direct = DIRECT_NO (never_direct allow)2016/12/29 16:12:41.844| 44,3| peer_select.cc(441) peerSelectFoo: GET 172.22.8.1452016/12/29 16:12:41.844| 44,3| peer_select.cc(110) peerSelectIcpPing: peerSelectIcpPing: http://172.22.8.145:3128/squid-internal-periodic/store_ digest 2016/12/29 16:12:41.844| 44,3| peer_select.cc(121) peerSelectIcpPing: peerSelectIcpPing: counted 0 neighbors2016/12/29 16:12:41.844| 44,3| peer_select.cc(685) peerGetSomeParent: GET 172.22.8.1452016/12/29 16:12:41.844| 44,3| peer_select.cc(709) peerGetSomeParent: peerSelect: FIRSTUP_PARENT/127.0.0.12016/12/29 16:12:41.844| 44,5| peer_select.cc(938) peerAddFwdServer: peerAddFwdServer: adding 127.0.0.1 FIRSTUP_PARENT2016/12/29 16:12:41.844| 44,5| peer_select.cc(938) peerAddFwdServer: peerAddFwdServer: adding 127.0.0.1 ANY_OLD_PARENT2016/12/29 16:12:41.844| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: http://172.22.8.145:3128/squid-internal-periodic/store_ ' via 127.0.0.1digest 2016/12/29 16:12:41.844| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: http://172.22.8.145:3128/squid-internal-periodic/store_ ' via 127.0.0.1digest 2016/12/29 16:12:41.844| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'http://172.22.8.145:3128/squid-internal-periodic/store_ 'digest 2016/12/29 16:12:41.844| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = DENIED2016/12/29 16:12:41.844| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = ALLOWED2016/12/29 16:12:41.844| 44,2| peer_select.cc(292) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=127.0.0.1:18070 flags=12016/12/29 16:12:41.844| 44,2| peer_select.cc(292) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=127.0.0.1:18070 flags=12016/12/29 16:12:41.844| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 02016/12/29 16:12:41.844| 44,3| peer_select.cc(79) ~ps_state: http://172.22.8.145:3128/squid-internal-periodic/store_ digest On Thu, Dec 29, 2016 at 2:21 PM, Ivan Larionov <xeron.oskom@xxxxxxxxx> wrote:Thank you for helping.After some experiments and tcpdumping it looks like it's not sibling sending request to the parent, but original squid!So instead of asking sibling about his digests squid asks parent.And your trick with urlpath_regex didn't help. I even tried:acl internal_digest urlpath_regex +i /.*store_digest.*/always_direct allow internal_digestnever_direct deny internal_digestbut no luck. It still asks parent.--On Thu, Dec 29, 2016 at 1:00 AM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote:On 2016-12-29 20:51, Ivan Larionov wrote:
I'm sure about forwarding because I see requests to
http://172.22.15.88:3128/squid-internal-periodic/store_diges [1] int
parent logs and my parent returns 502 because we do not allow requests
to internal IPs. Logs from the parent:
Got request: GET
http://172.22.15.88:3128/squid-internal-periodic/store_diges t
Not allowing blacklisted IP 172.22.15.88
GET http://172.22.15.88:3128/squid-internal-periodic/store_diges 502t
0ms
I do not have "global_internal_static off" in my config and also I'm
able to get
http://172.22.15.88:3128/squid-internal-periodic/store_diges [1]t
using curl or telnet (with telnet I do "GET
/squid-internal-periodic/store_digest" – note relative URL).
Okay, thats good.
However according to debug logs squid does this request using absolute
URL which probably works if target sibling can do direct requests (so
it will request itself for digest and return response to original
squid). But I do have "never_direct allow all" which probably makes
sibling to forward such request to a parent.
Hmm, I think you might be right about that.
You can test it by adding:
acl foo urlpath_regex +i /squid.internal.digest/
never_direct deny foo
If my theory about absolute vs relative URL is correct then I believe
original squid should make store_digest request using relative URL
(like I can do with telnet) so sibling squid will return response
right away w/o asking itself for result.
Whats happening with the URL is that the sending peer generates it from the cache_peer IP/host name and port.
The receiving peer checks the pathstarts with "/squid-internal-" and that the hostname portion matches its own visible_hostname or unique_hostname. If those match its marked for special handling as an internal request, otherwise global_internal_static is used to determine if the hostname not matching is ignored and it gets marked anyway.
Since the digest needs to be targeted at the specific peer and not anything which may inject itself in between them the hostname does need to be sent. The relative URLs are for things that don't vary between proxies, like the Squid icons.
If you configure cache_peer with the hostname of the receiving peer instead of its raw-IP the requests should be sent with that hostname instead of raw-IP.
The config looks okay. Thanks for that.
Amos
With best regards, Ivan Larionov.--With best regards, Ivan Larionov.
With best regards, Ivan Larionov.
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
