On 2023-09-27 15:43, Dmitry Katsubo wrote:
On 2023-09-27 19:15, Alex Rousskov wrote:
On 2023-09-27 11:08, Dmitry Katsubo wrote:
After upgrading Squid from v4.13-10+deb11u2 (bullseye) to v5.7-2
(bookworm) I started to get about 5 core dumps per day like below, provided .
How can I find out the root of the problem and eliminate it?
Your Squid is most likely suffering (among other v5 bugs) from Squid Bug
5154: https://bugs.squid-cache.org/show_bug.cgi?id=5154
There is currently no fix for that bug (that I can recommend without serious caveats). There has been significant progress towards a fix, but it may have stalled at
https://github.com/squid-cache/squid/pull/1421
The best known way to prevent bug 5154 is to enable IPv6 support, but that is not feasible in many environments.
Another Squid admin has reported success with an unofficial (and probably incomplete) workaround at https://lists.squid-cache.org/pipermail/squid-users/2023-September/026103.html
Debain sid has v6.1-2, which I believe is also affected, hence no reason to try it 🤔
Your call. I believe Squid v6 has fixed several bugs that are still
present in v5.
After reading the bug#5154 description I see that the issue is because of disabled IPv6, in my case, via sysctl:
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.all.disable_ipv6=1
I believe re-enabling IPV6 should help the issue?
If Squid successfully detects IPv6 support, then bug 5154 crashes should
go away. If Squid does not, it should log a message to cache.log that
starts with "WARNING: BCP 177 violation"
I have checked why I have disabled it ... and it turned out that was an advise to fix the following error message in /var/log/squid/cache.log:
SendEcho ERROR: sending to ICMPv6 packet to [2606:4700:e0::ac40:6225]: (101) Network is unreachable
😱 Boomerang has returned.
I have not studied that ICMP problem, but it might have a solution that
does not involve disabling IPv6 support. The latter, as it turns out,
opens a rather large Pandora box!
HTH,
Alex.
# coredumpctl gdb -1 --no-pager
PID: 357187 (squid)
UID: 13 (proxy)
GID: 13 (proxy)
Signal: 6 (ABRT)
Timestamp: Wed 2023-09-27 11:27:08 CEST (5h 16min ago)
Command Line: $'(squid-1)' --kid squid-1 --foreground -sYC
Executable: /usr/sbin/squid
Control Group: /system.slice/squid.service
Unit: squid.service
Slice: system.slice
Storage: /var/lib/systemd/coredump/core.squid.13.f7e894167d564f788825cae8827bd2df.357187.1695806828000000.zst (present)
Size on Disk: 2.5M
Message: Process 357187 (squid) of user 13 dumped core.
Module libsystemd.so.0 from deb systemd-252.12-1~deb12u1.amd64
Stack trace of thread 357187:
#0 0x00007f752b2a9d3c n/a (libc.so.6 + 0x8ad3c)
#1 0x00007f752b25af32 raise (libc.so.6 + 0x3bf32)
#2 0x00007f752b245472 abort (libc.so.6 + 0x26472)
#3 0x00007f752b245395 n/a (libc.so.6 + 0x26395)
#4 0x00007f752b253e32 __assert_fail (libc.so.6 + 0x34e32)
#5 0x00005576cf7c0ee4 _ZNK2Ip7Address11getAddrInfoERP8addrinfoi (squid + 0x466ee4)
#6 0x00005576cf7bc3c5 _Z11comm_openexiiRN2Ip7AddressEiPKc (squid + 0x4623c5)
#7 0x00005576cf80fdfe _ZN4Comm10ConnOpener8createFdEv (squid + 0x4b5dfe)
#8 0x00005576cf8109c1 _ZN4Comm10ConnOpener5startEv (squid + 0x4b69c1)
#9 0x00005576cf7ab09c _ZN9JobDialerI8AsyncJobE4dialER9AsyncCall (squid + 0x45109c)
#10 0x00005576cf7a779f _ZN14AsyncCallQueue8fireNextEv (squid + 0x44d79f)
#11 0x00005576cf7a7b52 _ZN14AsyncCallQueue4fireEv (squid + 0x44db52)
#12 0x00005576cf55f689 _ZN9EventLoop7runOnceEv (squid + 0x205689)
#13 0x00005576cf55f778 _ZN9EventLoop3runEv (squid + 0x205778)
#14 0x00005576cf665a50 _Z9SquidMainiPPc (squid + 0x30ba50)
#15 0x00005576cf4fdae1 main (squid + 0x1a3ae1)
#16 0x00007f752b2461ca n/a (libc.so.6 + 0x271ca)
#17 0x00007f752b246285 __libc_start_main (libc.so.6 + 0x27285)
#18 0x00005576cf505cc1 _start (squid + 0x1abcc1)
ELF object binary architecture: AMD x86-64
GNU gdb (Debian 13.1-3) 13.1
Reading symbols from /usr/sbin/squid...
(No debugging symbols found in /usr/sbin/squid)
[New LWP 357187]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `(squid-1) --kid squid-1 --foreground -sYC'.
Program terminated with signal SIGABRT, Aborted.
(gdb) bt
#0 0x00007f752b2a9d3c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f752b25af32 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f752b245472 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007f752b245395 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007f752b253e32 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x00005576cf7c0ee4 in Ip::Address::getAddrInfo (this=this@entry=0x5576d2407a98, dst=@0x7ffea5b87f90: 0x5576d2a8af00, force=force@entry=0) at ./src/ip/Address.cc:663
#6 0x00005576cf7bc3c5 in comm_openex (sock_type=sock_type@entry=1, proto=proto@entry=6, addr=..., flags=1, note=0x5576d2ab0cd0 "[2a03:2880:f273:c8:face:b00c:0:167]") at ./src/comm.cc:347
#7 0x00005576cf80fdfe in Comm::ConnOpener::createFd (this=this@entry=0x5576d2a86bc8) at ./src/comm/ConnOpener.cc:288
#8 0x00005576cf8109c1 in Comm::ConnOpener::start (this=0x5576d2a86bc8) at ./src/comm/ConnOpener.cc:261
#9 0x00005576cf7ab09c in JobDialer<AsyncJob>::dial (this=0x5576d1e16558, call=...) at ../../src/base/AsyncJobCalls.h:175
#10 0x00005576cf7a779f in AsyncCallQueue::fireNext (this=this@entry=0x5576d1abbae0) at ./src/base/AsyncCallQueue.cc:60
#11 0x00005576cf7a7b52 in AsyncCallQueue::fire (this=0x5576d1abbae0) at ./src/base/AsyncCallQueue.cc:43
#12 0x00005576cf55f689 in EventLoop::dispatchCalls (this=0x7ffea5b885a0) at ./src/EventLoop.cc:144
#13 EventLoop::runOnce (this=this@entry=0x7ffea5b885a0) at ./src/EventLoop.cc:121
#14 0x00005576cf55f778 in EventLoop::run (this=this@entry=0x7ffea5b885a0) at ./src/EventLoop.cc:83
#15 0x00005576cf665a50 in SquidMain (argc=argc@entry=5, argv=argv@entry=0x7ffea5b88888) at ./src/main.cc:1719
#16 0x00005576cf4fdae1 in SquidMainSafe (argv=0x7ffea5b88888, argc=5) at ./src/main.cc:1406
#17 main (argc=5, argv=0x7ffea5b88888) at ./src/main.cc:1394
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
