Anyone? On Wed, Jan 20, 2010 at 4:17 PM, myocella <myocella@xxxxxxxxx> wrote: > Greetings, > > A Squid server (RHEL-5.2 64-bit) is serving around 1000 users with > NTLM (winbindd), > Basic, wbinfo, and URL rewriter (Websense). Squid has randomly crashed with > the following messages: > > FATAL: Received Segment Violation...dying. > 2010/01/20 15:53:29| storeDirWriteCleanLogs: Starting... > 2010/01/20 15:53:29| WARNING: Closing open FD 183 > 2010/01/20 15:53:29| commSetEvents: epoll_ctl(EPOLL_CTL_DEL): failed > on fd=183: (1) Operation not permitted > 2010/01/20 15:53:29| 65536 entries written so far. > 2010/01/20 15:53:29| 131072 entries written so far. > ... > ... > 2010/01/20 15:53:35| Finished. Wrote 10511684 entries. > 2010/01/20 15:53:35| Took 6.2 seconds (1702503.8 entries/sec). > CPU Usage: 3406.086 seconds = 2431.073 user + 975.013 sys > Maximum Resident Size: 0 KB > Page faults with physical i/o: 3 > Memory usage for squid via mallinfo(): > total space in arena: -1092812 KB > Ordinary blocks: -1096748 KB 3986 blks > Small blocks: 0 KB 24 blks > Holding blocks: 24108 KB 8 blks > Free Small blocks: 1 KB > Free Ordinary blocks: 3935 KB > Total in use: -1072640 KB 100% > Total free: 3936 KB 0% > 2010/01/20 15:54:48| Starting Squid Cache version 2.7.STABLE7 for > x86_64-unknown-linux-gnu... > > > > Ran gdb over the core dump file and got this result: > > # gdb /usr/local/squid/sbin/squid core.30068 > GNU gdb Red Hat Linux (6.5-37.el5rh) > Copyright (C) 2006 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db > library "/lib64/libthread_db.so.1". > > Reading symbols from /lib64/libcrypt.so.1...done. > Loaded symbols for /lib64/libcrypt.so.1 > Reading symbols from /lib64/libpthread.so.0...done. > Loaded symbols for /lib64/libpthread.so.0 > Reading symbols from /lib64/libm.so.6...done. > Loaded symbols for /lib64/libm.so.6 > Reading symbols from /lib64/libnsl.so.1...done. > Loaded symbols for /lib64/libnsl.so.1 > Reading symbols from /lib64/libc.so.6...done. > Loaded symbols for /lib64/libc.so.6 > Reading symbols from /lib64/ld-linux-x86-64.so.2...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /lib64/libnss_files.so.2...done. > Loaded symbols for /lib64/libnss_files.so.2 > Reading symbols from /lib64/libnss_dns.so.2...done. > Loaded symbols for /lib64/libnss_dns.so.2 > Reading symbols from /lib64/libresolv.so.2...done. > Loaded symbols for /lib64/libresolv.so.2 > Core was generated by `(squid)'. > Program terminated with signal 6, Aborted. > #0 0x000000376b630215 in raise () from /lib64/libc.so.6 > (gdb) bt > #0 0x000000376b630215 in raise () from /lib64/libc.so.6 > #1 0x000000376b631cc0 in abort () from /lib64/libc.so.6 > #2 0x0000000000474515 in death (sig=<value optimized out>) at tools.c:327 > #3 <signal handler called> > #4 0x000000000044271a in httpHeaderGetEntry (hdr=0x141868f8, > pos=0x7fff2c609774) at HttpHeader.c:595 > #5 0x00000000004427fe in httpHeaderFindEntry (hdr=0x141868f8, > id=HDR_CONTENT_TYPE) at HttpHeader.c:619 > #6 0x0000000000443037 in httpHeaderGetStr (hdr=0x141868f8, > id=HDR_CONTENT_TYPE) at HttpHeader.c:1086 > #7 0x00000000004075b9 in aclMatchAclList (list=0x2aaac246fa40, > checklist=0x2aaad3524738) at acl.c:2010 > #8 0x0000000000407d82 in aclCheck (checklist=0x2aaad3524738) at acl.c:2178 > #9 0x000000000042daae in externalAclHandleReply (data=0x2aaab26015d0, > reply=0x2aaac6951e38 "") at external_acl.c:985 > #10 0x000000000043cd65 in helperHandleRead (fd=<value optimized out>, > data=<value optimized out>) at helper.c:769 > #11 0x0000000000427511 in comm_select (msec=<value optimized out>) > at comm_generic.c:264 > #12 0x000000000044f7a6 in main (argc=<value optimized out>, > argv=0x7fff2c609a58) at main.c:863 > > The configure options read > > ./configure --prefix=/usr/local/squid --enable-async-io --enable-snmp --enab > le-epoll --enable-storeio=aufs null --enable-removal-policies=heap lru > --enable- > large-cache-files --with-large-files --disable-poll --disable-select > --disable-c > arp --disable-wccp --disable-ident-lookups --with-maxfd=32768 > --enable-ntlm-auth > -helpers=SMB --enable-external-acl-helpers=wbinfo_group ldap_group > --enable-auth > =basic ntlm --enable-basic-auth-helpers=LDAP --enable-err-languages=English > > > Snippet of squid.conf > > emulate_httpd_log on > buffered_logs on > debug_options ALL,1 > > logfile_daemon /usr/local/squid/libexec/logfile-daemon > access_log daemon:/var/log/squid/access.log > > shutdown_lifetime 10 seconds > > cache_dir aufs /var/spool/squid/cache01 62500 32 256 > cache_dir aufs /var/spool/squid/cache02 62500 32 256 > cache_dir aufs /var/spool/squid/cache03 62500 32 256 > cache_dir aufs /var/spool/squid/cache04 62500 32 256 > > cache_replacement_policy heap GDSF > memory_replacement_policy heap GDSF > > cache_log /var/log/squid/cache.log > cache_store_log none > pid_filename /var/run/squid.pid > coredump_dir /var/spool/squid/crash > log_icp_queries off > client_db on > half_closed_clients off > > cache_mem 512 MB > maximum_object_size 768000 KB > maximum_object_size_in_memory 96 KB > memory_pools off > > forwarded_for off > > snmp_port 1601 > snmp_incoming_address 0.0.0.0 > snmp_outgoing_address 255.255.255.255 > > auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp > auth_param ntlm children 30 > auth_param ntlm keep_alive on > > auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic > auth_param basic children 5 > auth_param basic realm User Authentication > > external_acl_type ads-group children=20 %LOGIN > /usr/local/squid/libexec/wbinfo_group.pl > > url_rewrite_children 50 > redirector_bypass off > url_rewrite_program /opt/Websense/bin/WsRedtor > > > Does anyone have any idea how to fix this problem? > > Many Thanks > > myOcella >
