Hi,
Sorry for the title, but I really don't know how to describe what just happened today. It's really odd.
I previously posted a few similar issues which were all fixed if I increased certain parameters (ulimits, children-{max,startup,idle}, TTL, etc.).
This time however, after several days trouble-free I got another show-stopper. The local squid cache stopped serving for almost half an hour. After that, it all started working again magically. I had the chance to log into the server with ssh and try a few things:
- In the cache log I could see these messages:
Starting new bllookup helpers...
helperOpenServers: Starting 10/80 'squid_url_lookup.pl' processes
WARNING: Cannot run '/opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl' process.
WARNING: Cannot run '/opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl' process.
WARNING: Cannot run '/opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl' process.
It doesn't say much as to why it "cannot run" the external program.
This is how the program is defined in squid.conf:
external_acl_type bllookup ttl=86400 negative_ttl=86400 children-max=80 children-startup=40 children-idle=10 %URI /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl [...]
Other than that, the log is pretty quiet.
The HTTP clients do not get served at all. They keep waiting for a reply.
# ps aux | grep squid
root 3043 0.0 0.0 84856 1728 ? Ss Aug31 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.http.conf -n squidhttp
squid 3046 0.0 0.0 128232 31052 ? S Aug31 0:35 (squid-1) -YC -f /etc/squid/squid.http.conf -n squidhttp
root 3538 0.0 0.0 86912 1740 ? Ss Aug31 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.https.conf -n squidhttps
squid 3540 0.0 0.1 134616 35608 ? S Aug31 1:09 (squid-1) -YC -f /etc/squid/squid.https.conf -n squidhttps
root 5690 0.0 0.0 87444 1736 ? Ss Aug31 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid
squid 5694 2.4 6.5 3769624 2136968 ? S Aug31 293:24 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 5727 0.0 0.0 4008 524 ? S Aug31 0:01 (unlinkd)
squid 5728 0.0 0.0 13904 1576 ? S Aug31 2:09 diskd 5830660 5830661 5830662
squid 11927 0.0 0.0 4156 644 ? S Sep07 0:36 (logfile-daemon) /var/log/squid/access.log
squid 11937 1.7 0.0 41792 6232 ? S Sep07 31:08 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11939 0.1 0.0 41776 6288 ? S Sep07 3:09 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11940 0.0 0.0 41784 6356 ? S Sep07 0:28 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11941 0.0 0.0 41800 6308 ? S Sep07 0:07 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11942 0.0 0.0 41800 6308 ? S Sep07 0:02 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11943 0.0 0.0 41784 6320 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11944 0.0 0.0 41784 6068 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11945 0.0 0.0 41780 6372 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11946 0.0 0.0 41800 6852 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11947 0.0 0.0 41784 6756 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11948 0.0 0.0 41792 6784 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11949 0.0 0.0 41780 6672 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11950 0.0 0.0 41780 6660 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11951 0.0 0.0 41760 6308 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11952 0.0 0.0 41772 6336 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11953 0.0 0.0 41772 6284 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11954 0.0 0.0 41776 6956 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11955 0.0 0.0 41772 6524 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11956 0.0 0.0 41772 6664 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11958 0.0 0.0 41772 6284 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11959 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11960 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11968 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11969 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11970 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11971 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11972 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11973 0.0 0.0 40444 3528 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11974 0.0 0.0 40444 3364 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11977 0.0 0.0 40444 3528 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11979 0.0 0.0 40444 3528 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11980 0.0 0.0 40444 3528 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11981 0.0 0.0 40444 3532 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11982 0.0 0.0 40444 3528 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11983 0.0 0.0 40444 3524 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11984 0.0 0.0 40444 3528 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11986 0.0 0.0 40444 3364 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11987 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11988 0.0 0.0 40444 3336 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11989 0.0 0.0 40444 3368 ? S Sep07 0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid 11990 0.0 0.0 27204 8000 ? S Sep07 0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid 11991 0.0 0.0 27204 7744 ? S Sep07 0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid 11992 0.0 0.0 27140 7816 ? S Sep07 0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid 11993 0.0 0.0 27140 7824 ? S Sep07 0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid 11994 0.0 0.0 27140 7756 ? S Sep07 0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid 11995 0.1 0.0 69048 12904 ? S Sep07 3:32 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 11996 0.0 0.0 69052 12764 ? S Sep07 0:22 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 11997 0.0 0.0 69064 12796 ? S Sep07 0:07 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 11998 0.0 0.0 69056 13232 ? S Sep07 0:04 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 11999 0.0 0.0 69036 13108 ? S Sep07 0:03 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12000 0.0 0.0 69028 12944 ? S Sep07 0:02 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12001 0.0 0.0 69044 12824 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12003 0.0 0.0 68920 12716 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12004 0.0 0.0 68792 12596 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12005 0.0 0.0 68792 12408 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12006 0.0 0.0 68792 12712 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12007 0.0 0.0 68792 12716 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12008 0.0 0.0 68792 12532 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12009 0.0 0.0 68792 12664 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12010 0.0 0.0 68792 12504 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12011 0.0 0.0 68792 12660 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12012 0.0 0.0 68792 12556 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12013 0.0 0.0 68792 13816 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12014 0.0 0.0 68792 13740 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12015 0.0 0.0 68792 13756 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12016 0.0 0.0 68792 13820 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12017 0.0 0.0 68792 13812 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12018 0.0 0.0 68792 13820 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12019 0.0 0.0 68792 13792 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12020 0.0 0.0 32932 9372 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12021 0.0 0.0 32932 9396 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12022 0.0 0.0 32932 9364 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12023 0.0 0.0 32932 9400 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12025 0.0 0.0 32932 9396 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12026 0.0 0.0 32932 9396 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12027 0.0 0.0 32932 9372 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12028 0.0 0.0 32932 9368 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12029 0.0 0.0 32932 9372 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12032 0.0 0.0 32932 9372 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12033 0.0 0.0 32932 9368 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12034 0.0 0.0 32932 9392 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12036 0.0 0.0 32932 9404 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12037 0.0 0.0 32932 9400 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12038 0.0 0.0 32932 9368 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 12039 0.0 0.0 32932 9368 ? S Sep07 0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid 14108 0.0 0.0 24404 1392 ? S 02:13 0:00 (negotiate_kerberos_auth) -s HTTP/inf-fw2.mydomain.org@xxxxxxxxxxxx
root 28492 0.0 0.0 86908 4220 ? Ss Sep05 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa.conf -n squidowa
squid 28495 0.0 0.0 103616 18364 ? S Sep05 0:12 (squid-1) -YC -f /etc/squid/squid.owa.conf -n squidowa
root 29120 0.0 0.0 86908 4220 ? Ss Sep05 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid 29123 0.2 0.6 293460 206776 ? S Sep05 10:30 (squid-1) -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid 30291 0.0 0.0 24404 2392 ? S Sep07 0:00 (negotiate_kerberos_auth) -s HTTP/inf-fw2.mydomain.org@xxxxxxxxxxxx
squid 30330 0.0 6.4 3769624 2127928 ? S 13:42 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 30866 0.0 6.4 3769624 2127928 ? S 13:44 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
top - 13:52:38 up 9 days, 6:19, 2 users, load average: 2.04, 1.82, 1.65
Tasks: 405 total, 1 running, 404 sleeping, 0 stopped, 0 zombie
%Cpu0 : 0.0 us, 0.0 sy, 0.0 ni, 97.4 id, 0.0 wa, 0.0 hi, 2.6 si, 0.0 st
%Cpu1 : 0.0 us, 0.3 sy, 0.0 ni, 97.0 id, 0.0 wa, 0.0 hi, 2.6 si, 0.0 st
%Cpu2 : 0.3 us, 0.3 sy, 0.0 ni, 98.7 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu3 : 0.0 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
%Cpu4 : 0.0 us, 0.0 sy, 0.0 ni, 96.7 id, 0.0 wa, 0.0 hi, 3.3 si, 0.0 st
%Cpu5 : 0.0 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu6 : 0.3 us, 0.0 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu7 : 0.3 us, 0.0 sy, 0.0 ni, 95.7 id, 0.0 wa, 0.0 hi, 4.0 si, 0.0 st
KiB Mem : 32865056 total, 12324092 free, 16396808 used, 4144156 buff/cache
KiB Swap: 37036988 total, 35197252 free, 1839736 used. 15977208 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29123 squid 20 0 293460 206776 8480 S 0.7 0.6 10:31.16 squid
427 root 20 0 20280 2912 2168 R 0.3 0.0 0:00.05 top
7902 named 20 0 821424 111644 5720 S 0.3 0.3 95:30.99 named
16766 suricata 20 0 2127364 358940 7356 S 0.3 1.1 3:12.03 Suricata-Main
1 root 20 0 4176 1512 1464 S 0.0 0.0 0:06.34 init
I then issued:
# strace -o squid.trace /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid
I didn't get much out of it, but I can post it if someone would like to see it.
I tried to stop Squid with my openrc script and failed.
So I issued the following manually several times:
# squid -n squid -k shutdown
The first result was:
# squid -n squid -k shutdown
2017/09/08 13:56:01.125| 24,8| SBuf.cc(124) ~SBuf: SBuf59 destructed
2017/09/08 13:56:01.125| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd74290 id=blob170 capacity=40 size=16
2017/09/08 13:56:01.125| 24,8| SBuf.cc(124) ~SBuf: SBuf48 destructed
2017/09/08 13:56:01.125| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd709a0 id=blob62 capacity=40 size=30
2017/09/08 13:56:01.125| 24,8| SBuf.cc(124) ~SBuf: SBuf44 destructed
2017/09/08 13:56:01.125| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xcdc6e0 id=blob51 capacity=40 size=12
2017/09/08 13:56:01.125| 28,3| Acl.cc(384) ~ACL: freeing ACL adaptation_access
2017/09/08 13:56:01.125| 45,9| cbdata.cc(321) cbdataInternalFree: 0xdf8298
2017/09/08 13:56:01.125| 45,9| cbdata.cc(338) cbdataInternalFree: Freeing 0xdf8298
2017/09/08 13:56:01.125| 24,8| SBuf.cc(124) ~SBuf: SBuf38 destructed
2017/09/08 13:56:01.125| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd60be0 id=blob37 capacity=16388 size=8
2017/09/08 13:56:01.125| 24,8| SBuf.cc(124) ~SBuf: SBuf37 destructed
2017/09/08 13:56:01.125| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd5cb90 id=blob36 capacity=16388 size=5
[...etc...]
Subsequent calls yielded:
squid: ERROR: Could not send signal 15 to process 5694: (3) No such process
2017/09/08 13:56:09.993| 24,8| SBuf.cc(124) ~SBuf: SBuf59 destructed
2017/09/08 13:56:09.993| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd74290 id=blob170 capacity=40 size=16
2017/09/08 13:56:09.993| 24,8| SBuf.cc(124) ~SBuf: SBuf48 destructed
2017/09/08 13:56:09.993| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd709a0 id=blob62 capacity=40 size=30
2017/09/08 13:56:09.993| 24,8| SBuf.cc(124) ~SBuf: SBuf44 destructed
2017/09/08 13:56:09.993| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xcdc6e0 id=blob51 capacity=40 size=12
2017/09/08 13:56:09.993| 28,3| Acl.cc(384) ~ACL: freeing ACL adaptation_access
2017/09/08 13:56:09.993| 45,9| cbdata.cc(321) cbdataInternalFree: 0xdf8298
2017/09/08 13:56:09.993| 45,9| cbdata.cc(338) cbdataInternalFree: Freeing 0xdf8298
2017/09/08 13:56:09.993| 24,8| SBuf.cc(124) ~SBuf: SBuf38 destructed
2017/09/08 13:56:09.993| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd60be0 id=blob37 capacity=16388 size=8
2017/09/08 13:56:09.993| 24,8| SBuf.cc(124) ~SBuf: SBuf37 destructed
2017/09/08 13:56:09.994| 24,9| MemBlob.cc(83) ~MemBlob: destructed, this=0xd5cb90 id=blob36 capacity=16388 size=5
[...etc...]
# ps aux | grep squid | grep '\-n'
squid 805 0.0 6.4 3769624 2127988 ? S 13:53 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 1452 0.0 6.4 3769624 2127988 ? S 13:55 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
root 3043 0.0 0.0 84856 1728 ? Ss Aug31 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.http.conf -n squidhttp
squid 3046 0.0 0.0 128232 31052 ? S Aug31 0:35 (squid-1) -YC -f /etc/squid/squid.http.conf -n squidhttp
root 3538 0.0 0.0 86912 1740 ? Ss Aug31 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.https.conf -n squidhttps
squid 3540 0.0 0.1 134940 36244 ? S Aug31 1:09 (squid-1) -YC -f /etc/squid/squid.https.conf -n squidhttps
root 28492 0.0 0.0 86908 4220 ? Ss Sep05 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa.conf -n squidowa
squid 28495 0.0 0.0 103616 18364 ? S Sep05 0:12 (squid-1) -YC -f /etc/squid/squid.owa.conf -n squidowa
root 29120 0.0 0.0 86908 4220 ? Ss Sep05 0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid 29123 0.2 0.6 293460 206776 ? S Sep05 10:31 (squid-1) -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid 30330 0.0 6.4 3769624 2127928 ? S 13:42 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 30866 0.0 6.4 3769624 2127928 ? S 13:44 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 31507 0.0 6.4 3769624 2127928 ? S 13:47 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 32055 0.0 6.4 3769624 2127928 ? S 13:49 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid 32659 0.0 6.4 3769624 2127928 ? S 13:51 0:00 (squid-1) -YC -f /etc/squid/squid.conf -n squid
I changed the debug_options to ALL,9 and tried to restart or "reconfigure" Squid.
I had trouble with that and decided to:
- stop all squid instances (the other reverse proxies, eg. squidhttp, squidhttps, squidowa, squidowa2, were working fine)
- manually kill all squid processes related to the failing local cache (including ssl_crtd)
- made sure there were no more squid processes and that debug_options was ALL,9
- started the local squid cache only (only 1 daemon - no reverse proxies)
Surprisingly, it still didn't work...
Clients could not browse. The squid log only had this set of messages once in a while (very quiet log):
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(180) ipcCreate: ipcCreate: prfd FD 94
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(181) ipcCreate: ipcCreate: pwfd FD 94
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(182) ipcCreate: ipcCreate: crfd FD 93
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(183) ipcCreate: ipcCreate: cwfd FD 93
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(196) ipcCreate: ipcCreate: FD 94 sockaddr [::1]:40905
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(212) ipcCreate: ipcCreate: FD 93 sockaddr [::1]:50647
2017/09/08 14:18:10.322 kid1| 54,3| ipc.cc(222) ipcCreate: ipcCreate: FD 93 listening...
2017/09/08 14:18:10.322 kid1| 5,3| comm.cc(868) _comm_close: comm_close: start closing FD 93
2017/09/08 14:18:10.322 kid1| 5,3| comm.cc(540) commUnsetFdTimeout: Remove timeout for FD 93
2017/09/08 14:18:10.322 kid1| 5,5| comm.cc(721) commCallCloseHandlers: commCallCloseHandlers: FD 93
2017/09/08 14:18:10.322 kid1| 5,4| AsyncCall.cc(26) AsyncCall: The AsyncCall comm_close_complete constructed, this=0xeb7c10 [call123]
2017/09/08 14:18:10.322 kid1| 5,4| AsyncCall.cc(93) ScheduleCall: comm.cc(941) will call comm_close_complete(FD 93) [call123]
2017/09/08 14:18:10.322 kid1| 5,9| comm.cc(602) comm_connect_addr: connecting socket FD 94 to [::1]:50647 (want family: 10)
2017/09/08 14:18:10.322 kid1| 21,3| tools.cc(543) leave_suid: leave_suid: PID 8303 called
2017/09/08 14:18:10.323 kid1| 21,3| tools.cc(636) no_suid: no_suid: PID 8303 giving up root priveleges forever
2017/09/08 14:18:10.323 kid1| 54,3| ipc.cc(304) ipcCreate: ipcCreate: calling accept on FD 93
[...]
2017/09/08 14:24:49.682 kid1| 5,5| comm.cc(644) comm_connect_addr: sock=98, addrinfo( flags=4, family=10, socktype=1, protocol=6, &addr=0xeb79a0, addrlen=28 )
2017/09/08 14:24:49.682 kid1| 5,9| comm.cc(645) comm_connect_addr: connect FD 98: (-1) (110) Connection timed out
2017/09/08 14:24:49.682 kid1| 14,9| comm.cc(646) comm_connect_addr: connecting to: [::1]:60557
2017/09/08 14:24:49.682 kid1| 5,3| comm.cc(868) _comm_close: comm_close: start closing FD 98
2017/09/08 14:24:49.682 kid1| 5,3| comm.cc(540) commUnsetFdTimeout: Remove timeout for FD 98
2017/09/08 14:24:49.682 kid1| 5,5| comm.cc(721) commCallCloseHandlers: commCallCloseHandlers: FD 98
2017/09/08 14:24:49.682 kid1| 5,4| AsyncCall.cc(26) AsyncCall: The AsyncCall comm_close_complete constructed, this=0xeb7e90 [call128]
2017/09/08 14:24:49.682 kid1| 5,4| AsyncCall.cc(93) ScheduleCall: comm.cc(941) will call comm_close_complete(FD 98) [call128]
2017/09/08 14:24:49.682 kid1| WARNING: Cannot run '/usr/libexec/squid/ext_wbinfo_group_acl' process.
2017/09/08 14:24:49.682 kid1| 50,3| comm.cc(347) comm_openex: comm_openex: Attempt open socket for: [::1]
2017/09/08 14:24:49.682 kid1| 50,3| comm.cc(388) comm_openex: comm_openex: Opened socket local=[::1] remote=[::] FD 99 flags=1 : family=10, type=1, protocol=0
2017/09/08 14:24:49.682 kid1| 5,5| comm.cc(420) comm_init_opened: local=[::1] remote=[::] FD 99 flags=1 is a new socket
2017/09/08 14:24:49.682 kid1| 51,3| fd.cc(198) fd_open: fd_open() FD 99 ext_wbinfo_group_acl
2017/09/08 14:24:49.682 kid1| 50,6| comm.cc(209) commBind: commBind: bind socket FD 99 to [::1]
2017/09/08 14:24:49.682 kid1| 50,3| comm.cc(347) comm_openex: comm_openex: Attempt open socket for: [::1]
2017/09/08 14:24:49.682 kid1| 50,3| comm.cc(388) comm_openex: comm_openex: Opened socket local=[::1] remote=[::] FD 100 flags=1 : family=10, type=1, protocol=0
2017/09/08 14:24:49.682 kid1| 5,5| comm.cc(420) comm_init_opened: local=[::1] remote=[::] FD 100 flags=1 is a new socket
2017/09/08 14:24:49.682 kid1| 51,3| fd.cc(198) fd_open: fd_open() FD 100 ext_wbinfo_group_acl
2017/09/08 14:24:49.682 kid1| 50,6| comm.cc(209) commBind: commBind: bind socket FD 100 to [::1]
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(180) ipcCreate: ipcCreate: prfd FD 100
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(181) ipcCreate: ipcCreate: pwfd FD 100
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(182) ipcCreate: ipcCreate: crfd FD 99
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(183) ipcCreate: ipcCreate: cwfd FD 99
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(196) ipcCreate: ipcCreate: FD 100 sockaddr [::1]:56367
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(212) ipcCreate: ipcCreate: FD 99 sockaddr [::1]:35643
2017/09/08 14:24:49.682 kid1| 54,3| ipc.cc(222) ipcCreate: ipcCreate: FD 99 listening...
2017/09/08 14:24:49.683 kid1| 5,3| comm.cc(868) _comm_close: comm_close: start closing FD 99
2017/09/08 14:24:49.683 kid1| 5,3| comm.cc(540) commUnsetFdTimeout: Remove timeout for FD 99
2017/09/08 14:24:49.683 kid1| 5,5| comm.cc(721) commCallCloseHandlers: commCallCloseHandlers: FD 99
2017/09/08 14:24:49.683 kid1| 5,4| AsyncCall.cc(26) AsyncCall: The AsyncCall comm_close_complete constructed, this=0xeb7f10 [call129]
2017/09/08 14:24:49.683 kid1| 5,4| AsyncCall.cc(93) ScheduleCall: comm.cc(941) will call comm_close_complete(FD 99) [call129]
2017/09/08 14:24:49.683 kid1| 5,9| comm.cc(602) comm_connect_addr: connecting socket FD 100 to [::1]:35643 (want family: 10)
2017/09/08 14:24:49.683 kid1| 21,3| tools.cc(543) leave_suid: leave_suid: PID 10064 called
2017/09/08 14:24:49.683 kid1| 21,3| tools.cc(636) no_suid: no_suid: PID 10064 giving up root priveleges forever
2017/09/08 14:24:49.683 kid1| 54,3| ipc.cc(304) ipcCreate: ipcCreate: calling accept on FD 99
I also tried restarting Winbind, c-icap server, clamd, but nothing changed. They all seemed to be working properly anyway.
Finally, at some point (after half an hour) right after killing all squid processes yet again, I restarted the squid services one last time (reverted to debug_options ALL,1) before giving up and deciding to let users bypass the proxy. Well, it all started working again...
So now I'd really like to know what I can do the next time it stops working like this.
I'm considering setting debug_options to ALL,6 while I still can, and wait to see if it fails again. When it does, I might have more information.
Any suggestions?
Vieri
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
