Squid-users, We have plenty of servers running all kinds of versions of Squid but one of the most stable version we run on a fast server, keeps crashing for an unknown reason. We hope someone on this mailing list can shed some light on the issue. About once a day (not every day) Squid crashes, makes a core dump and restarts. Everything else on the server experiences no problems, nothing odd in the systems log files and the memory came out just fine in a memtest. We didn't even notice it until we saw the diskspace usage grow with a few spikes. The core dumps were about 5 gigabytes each, the latest just 500 megabytes but they all show the same backtrace. Server information: Squid version: 2.7.STABLE6-2 Kernel: 2.6.24-24-server Issue: Ubuntu 8.04.3 LTS Memory: 12G CPU: single QC Intel(R) Xeon(R) CPU L5506 @ 2.13GHz Regular messages in the cache log (garbage has been edited out in all of the following pieces): """ httpReadReply: Excess data from "GET http://...."; """ """ storeAddVaryReadOld: New index aborted at 8188 (836) """ """ storeAufsOpenDone: (2) No such file or directory """ """ VaryData: accept-encoding="gzip,%20deflate", user-agent="Mozilla%2F4.0%20(compatible . %20CLR%203.0.30618)" """ Cache log right before the crash/restart: """ storeLocateVaryRead: Unexpected data ' . ' . Starting Squid Cache version 2.7.STABLE6 for amd64-debian-linux-gnu... """ gdb backtrace on the core dump: """ (gdb) backtrace #0 0x00007fdec9826095 in raise () from /lib/libc.so.6 #1 0x00007fdec9827af0 in abort () from /lib/libc.so.6 #2 0x00007fdec981f2df in __assert_fail () from /lib/libc.so.6 #3 0x00000000004ac124 in xstrndup ( s=0x16377744 "\nÙ\206A/\210\216p4< . \003\026ô", n=0) at util.c:616 #4 0x0000000000477ae4 in storeLocateVaryRead (data=0x1a08fc48, buf=0x1e6ebe00 "гj\036", size=4050) at store.c:878 #5 0x0000000000478274 in storeClientCallback (sc=0x12f7d428, sz=4050) at store_client.c:146 #6 0x000000000048fd6b in storeAufsReadDone (fd=<value optimized out>, my_data=0x140c8e58, buf=0x12f15ef0 "\215F\004\"\t\234G\b . 203ôõ\023 2º", len=<value optimized out>, errflag=0) at aufs/store_io_aufs.c:400 #7 0x0000000000491f4a in aioCheckCallbacks (SD=<value optimized out>) at aufs/async_io.c:319 #8 0x000000000047a7d8 in storeDirCallback () at store_dir.c:511 #9 0x000000000042a7ac in comm_select (msec=307) at comm_generic.c:377 #10 0x00000000004573e6 in main (argc=<value optimized out>, argv=0x7fffd28695d8) at main.c:884 """ Thijs Stuurman System Administrator Nxs Internet BV Kabelweg 37 1014 BA AMSTERDAM Tel.: +31 (0) 20 46 88 071 Fax: +31 (0) 20 46 88 236 Mail: beheer.linux@xxxxxx
