On 18/05/2012 6:45 a.m., Pablo Arturi wrote:
I'm having squid crash/restarting automatically from a week ago more
or less, without any hardware or software change.
I had the squid 3.1.16 release which I updated yesterday to 3.1.19
from the gentoo package manager, and it keep logging the same errors:
This is the evidence:
firewall ~ # cat /var/log/kern.log | grep squid
May 16 16:49:24 localhost kernel: squid[13702]: segfault at 0 ip
000000000053eae3 sp 000003df3c6c39b8 error 4 in squid[400000+23e000]
May 16 16:49:24 localhost kernel: grsec: From 10.200.145.55:
Segmentation fault occurred at (null) in /usr/sbin/squid[squid:13702]
uid/euid:0/31 gid/egid:31/31, parent /usr/sbin squid[squid:12267]
uid/euid:0/0 gid/egid:31/31
May 16 16:49:28 localhost kernel: squid[14162]: segfault at 0 ip
000000000053eae3 sp 000003a1d7901b98 error 4 in squid[400000+23e000]
May 16 16:49:28 localhost kernel: grsec: From 10.200.145.55:
Segmentation fault occurred at (null) in /usr/sbin/squid[squid:14162]
uid/euid:0/31 gid/egid:31/31, parent /usr/sbin squid[squid:12267]
uid/euid:0/0 gid/egid:31/31
firewall ~ # cat /var/log/messages | grep squid
May 16 16:49:24 localhost squid[12267]: Squid Parent: child process
13702 exited due to signal 11 with status 0
May 16 16:49:27 localhost squid[12267]: Squid Parent: child process
14162 started
May 16 16:49:28 localhost squid[12267]: Squid Parent: child process
14162 exited due to signal 11 with status 0
May 16 16:49:31 localhost squid[12267]: Squid Parent: child process
14201 started
<snip>
Any help will be appreciated.
Can you find or make a core dump to see where this crash is occuring?
There may also be some extra information in cache.log from Squid when
it exited.
http://wiki.squid-cache.org/SquidFaq/BugReporting
Amos
Amos, thanks for your reply.
This is the debug information, I took it from the suggested method in
SquidFaq/BugReporting which was creating a script with the following
code:
trap "rm -f $$.gdb" 0
cat <<EOF >$$.gdb
handle SIGPIPE pass nostop noprint
handle SIGTERM pass nostop noprint
handle SIGUSR1 pass nostop noprint
handle SIGSEGV stop
handle SIGABRT stop
run -DNYCd3
backtrace
quit
EOF
while sleep 2; do
gdb -x $$.gdb /path/to/squid 2>&1 | tee -a squid.out
doneThis is the squid.out content after some crashes (it crashed a few
times, but I think it's the same information, so I'll just put once;
if you need it I'll post the entire file to an URL so you can check
it).firewall ~ # cat /root/squid.out | less
GNU gdb (Gentoo 7.3.1 p2) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>...
Reading symbols from /usr/sbin/squid...(no debugging symbols
found)...done.
2012/05/17 13:48:52| WARNING: -D command-line option is obsolete.
2012/05/17 13:48:52| Starting Squid Cache version 3.1.19 for
x86_64-pc-linux-gnu...
2012/05/17 13:48:52| Process ID 16781
2012/05/17 13:48:52| With 1024 file descriptors available
2012/05/17 13:48:52| Initializing IP Cache...
2012/05/17 13:48:52| DNS Socket created at [::], FD 10
2012/05/17 13:48:52| DNS Socket created at 0.0.0.0, FD 11
2012/05/17 13:48:52| Adding nameserver 127.0.0.1 from squid.conf
2012/05/17 13:48:52| Adding nameserver 208.67.222.222 from squid.conf
2012/05/17 13:48:52| Adding nameserver 208.67.220.220 from squid.conf
2012/05/17 13:48:52| helperOpenServers: Starting 20/20 'squidGuard'
processes
2012/05/17 13:48:52| Referer logging is disabled.
2012/05/17 13:48:52| Unlinkd pipe opened on FD 57
2012/05/17 13:48:52| Local cache digest enabled; rebuild/rewrite every
3600/3600 sec
2012/05/17 13:48:52| Store logging disabled
2012/05/17 13:48:52| Swap maxSize 16777216 + 524288 KB, estimated
1330884 objects
2012/05/17 13:48:52| Target number of buckets: 66544
2012/05/17 13:48:52| Using 131072 Store buckets
2012/05/17 13:48:52| Max Mem size: 524288 KB
2012/05/17 13:48:52| Max Swap size: 16777216 KB
2012/05/17 13:48:52| Version 1 of swap file with LFS support detected...
2012/05/17 13:48:52| Rebuilding storage in /var/cache/squid (CLEAN)
2012/05/17 13:48:52| Using Least Load store dir selection
2012/05/17 13:48:52| Current Directory is /root
2012/05/17 13:48:52| Loaded Icons.
2012/05/17 13:48:52| Accepting intercepted HTTP connections at
127.0.0.1:3128, FD 60.
2012/05/17 13:48:52| Accepting intercepted HTTP connections at
10.200.145.100:3128, FD 61.
2012/05/17 13:48:52| Accepting intercepted HTTP connections at
50.50.50.213:3128, FD 62.
2012/05/17 13:48:52| HTCP Disabled.
2012/05/17 13:48:52| Squid plugin modules loaded: 0
2012/05/17 13:48:52| Ready to serve requests.
2012/05/17 13:48:52| Store rebuilding is 8.92% complete
2012/05/17 13:48:52| Done reading /var/cache/squid swaplog (45906
entries)
2012/05/17 13:48:52| Finished rebuilding storage from disk.
2012/05/17 13:48:52| 45906 Entries scanned
2012/05/17 13:48:52| 0 Invalid entries.
2012/05/17 13:48:52| 0 With invalid flags.
2012/05/17 13:48:52| 45906 Objects loaded.
2012/05/17 13:48:52| 0 Objects expired.
2012/05/17 13:48:52| 0 Objects cancelled.
2012/05/17 13:48:52| 0 Duplicate URLs purged.
2012/05/17 13:48:52| 0 Swapfile clashes avoided.
2012/05/17 13:48:52| Took 0.30 seconds (151289.75 objects/sec).
2012/05/17 13:48:52| Beginning Validation Procedure
2012/05/17 13:48:52| Completed Validation Procedure
2012/05/17 13:48:52| Validated 91837 Entries
2012/05/17 13:48:52| store_swap_size = 1045568
2012/05/17 13:48:53| storeLateRelease: released 0 objects
2012/05/17 14:12:08| squidaio_queue_request: WARNING - Queue congestion
2012/05/17 14:28:40| WARNING: swapfile header inconsistent with
available data
Program received signal SIGSEGV, Segmentation fault.
0x000000000053eae3 in storeRead ()
#0 0x000000000053eae3 in storeRead ()
#1 0x000000000053fbe9 in store_client::fileRead() ()
#2 0x00000000005a1418 in UFSStoreState::readCompleted(char const*,
int, int, RefCount<ReadRequest>) ()
This is being tracked in http://bugs.squid-cache.org/show_bug.cgi?id=3549
Thanks for the info this is being seen in 3.1.19.
Amos
