IT seems that things are doing good with out huge domain list. so now my next goal is squidguard. but the problem with squid guard was that i tried it configuring and i saw many online manuals but it didnt activated so i just started using domain list. however if thing doesnt work ill update the status. Thanks you all for your kind help. Thanks On Fri, Apr 27, 2012 at 1:09 PM, Muhammad Yousuf Khan <sirtcp@xxxxxxxxx> wrote: > i think the delay was due to the 10mb of domain list. it seems that > things are back to track now. however for further restriction ill look > into other solutions as suggested in this thread. > > Thanks alot > > On Wed, Apr 25, 2012 at 9:13 PM, Muhammad Yousuf Khan <sirtcp@xxxxxxxxx> wrote: >> Thanks , i learn some thing new from you all. however ill update the >> results in few days as i am monitoring the stuff as how things are >> going. >> >> Thanks, >> >> On Wed, Apr 25, 2012 at 7:38 AM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote: >>> On 25/04/2012 3:34 a.m., Eliezer Croitoru wrote: >>>> >>>> On 24/04/2012 18:14, Muhammad Yousuf Khan wrote: >>>>> >>>>> ok i trim down config file to this as you suggested of blocking >>>>> whitelist to local net.. let see how things work tommorw. ill update. >>>>> but block list is like 10MB big do you think it could be the >>>>> problem.as every query has to be matched with 10 MB database. >>>>> >>>>> ? >>>> >>>> in any case a dstdomain of 10MB is a very bad idea from what i know. >>>> one thing about dstdomain is that squid must validate the request dns >>>> records and it will take more bandwidth on dns queries. >>> >>> >>> Only if comparing a raw-IP to a domain name. If the raw-IP is on teh tested >>> URL it is faster as the DNS result gets re-used for all tests. The common >>> case though is straight domain-vs-domain comparisons. >>> >>> Amos >>> >>> >>>> if you still dont have local dns server for cahing only this is the time >>>> to add it. >>>> >>>> i think that 10MB of domains can be optimized into some basic DST DOMAINS >>>> REGEX and some blacklist DSTDOMS REGEX. >>>> >>>> i think that some db application for this kind of amount of dstdoms can >>>> much more effective. >>>> you can also use squidguard for that. >>>> >>>> if you can share some (1MB) of the dstdoms of the whole list i might be >>>> able to try to optimize it in a way. >>>> >>>> >>>> Regards, >>>> Eliezer >>>> >>>>> >>>>> >>>>> >>>>> #-------------Allow All ACL------------- >>>>> acl aci_lan src 10.51.100.0/24 >>>>> acl aci_general src 10.51.100.0/24 >>>>> >>>>> #---------------------Assurety Whitelist--------------- >>>>> acl aci_whitelist dstdomain "/blocklist/aci_list/whitelist" >>>>> http_access allow aci_whitelist aci_general >>>>> >>>>> #----------TimeDomainBlock >>>>> acl aci_dest dstdomain "/blocklist/aci_list/time_block_domains" >>>>> >>>>> #--General Timing------------ Normal Days Working hours-------------- >>>>> acl aci_working_hours time MTWH 10:04-13:04 >>>>> acl aci_working_hours time MTWH 14:04-18:04 >>>>> #--General Timing-------------Friday------------------------ >>>>> acl aci_working_hours time F 10:04-13:04 >>>>> acl aci_working_hours time F 15:04-18:04 >>>>> >>>>> http_access deny aci_dest aci_working_hours aci_general >>>>> >>>>> >>>>> On Tue, Apr 24, 2012 at 1:11 PM, Eliezer Croitoru<eliezer@xxxxxxxxxxxx> >>>>> wrote: >>>>>> >>>>>> are you taking about the delay pools rules? >>>>>> also if it's a proxy that is open to the internet i would limit the >>>>>> access >>>>>> to port 3128 to only lan. >>>>>> your http_access rules are allowing anyone to use the proxy for the >>>>>> whitelist. >>>>>> >>>>>> Regards, >>>>>> Eliezer >>>>>> >>>>>> >>>>>> >>>>>> On 24/04/2012 09:06, Muhammad Yousuf Khan wrote: >>>>>>> >>>>>>> >>>>>>> ok i just disabled all the rules and it works for me now ill test >>>>>>> which rule is making a problem and let you know also. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Mon, Apr 23, 2012 at 11:20 PM, Muhammad Yousuf >>>>>>> Khan<sirtcp@xxxxxxxxx> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> here is the log for bbc.co.uk . first and last msg of log >>>>>>>> >>>>>>>> so you can see the time delay. >>>>>>>> >>>>>>>> 335205033.183 841 10.51.100.240 TCP_MISS/200 24506 GET >>>>>>>> http://www.bbc.co.uk/ - DIRECT/212.58.244.66 text/html >>>>>>>> 1335205057.936 328 10.51.100.240 TCP_REFRESH_HIT/304 435 GET >>>>>>>> >>>>>>>> http://static.bbci.co.uk/wwhomepage-3.5/1.0.41/img/broadcast-sprite.png >>>>>>>> - DIRECT/80.239.148.70 image/png >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Apr 23, 2012 at 11:12 PM, Muhammad Yousuf >>>>>>>> Khan<sirtcp@xxxxxxxxx> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Here you go with my squid.conf >>>>>>>>> >>>>>>>>> acl all src all >>>>>>>>> acl manager proto cache_object >>>>>>>>> acl localhost src 127.0.0.1/32 >>>>>>>>> acl to_localhost dst 127.0.0.0/8 >>>>>>>>> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network >>>>>>>>> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network >>>>>>>>> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network >>>>>>>>> acl SSL_ports port 443 # https >>>>>>>>> acl SSL_ports port 563 # snews >>>>>>>>> acl SSL_ports port 873 # rsync >>>>>>>>> acl Safe_ports port 80 # http >>>>>>>>> acl Safe_ports port 21 # ftp >>>>>>>>> acl Safe_ports port 443 # https >>>>>>>>> acl Safe_ports port 70 # gopher >>>>>>>>> acl Safe_ports port 210 # wais >>>>>>>>> acl Safe_ports port 1025-65535 # unregistered ports >>>>>>>>> acl Safe_ports port 280 # http-mgmt >>>>>>>>> acl Safe_ports port 488 # gss-http >>>>>>>>> acl Safe_ports port 591 # filemaker >>>>>>>>> acl Safe_ports port 777 # multiling http >>>>>>>>> acl Safe_ports port 631 # cups >>>>>>>>> acl Safe_ports port 873 # rsync >>>>>>>>> acl Safe_ports port 901 # SWAT >>>>>>>>> acl purge method PURGE >>>>>>>>> acl CONNECT method CONNECT >>>>>>>>> >>>>>>>>> # sqstat >>>>>>>>> acl manager proto cache_object >>>>>>>>> acl webserver src 10.51.100.206/255.255.255.255 >>>>>>>>> http_access allow manager webserver >>>>>>>>> http_access deny manager >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> # Skype >>>>>>>>> acl numeric_IPs dstdom_regex >>>>>>>>> >>>>>>>>> >>>>>>>>> ^(([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)|(\[([0-9af]+)?:([0-9af:]+)?:([0-9af]+)?\])):443 >>>>>>>>> acl Skype_UA browser ^skype >>>>>>>>> acl validUserAgent browser \S+ >>>>>>>>> >>>>>>>>> # for cheetah only >>>>>>>>> >>>>>>>>> #acl usman src 10.51.100.107 >>>>>>>>> #delay_pools 1 >>>>>>>>> #delay_class 1 1 >>>>>>>>> #delay_parameters 1 22000/22000 >>>>>>>>> #delay_access 1 allow usman >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> #-------------Allow All ACL------------- >>>>>>>>> acl aci_lan src 10.51.100.0/24 >>>>>>>>> acl aci_general src 10.51.100.0/24 >>>>>>>>> >>>>>>>>> >>>>>>>>> #----My ip >>>>>>>>> acl my_ip src 10.51.100.240 >>>>>>>>> http_access allow my_ip >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> # Testing delay pool >>>>>>>>> delay_pools 1 >>>>>>>>> delay_class 1 1 >>>>>>>>> delay_parameters 1 22000/10240000 >>>>>>>>> delay_access 1 allow aci_general >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> #---------------------Assurety Whitelist--------------- >>>>>>>>> acl aci_whitelist dstdomain "/blocklist/aci_list/whitelist" >>>>>>>>> http_access allow aci_whitelist >>>>>>>>> >>>>>>>>> #--Senior Allow Domainlist------------------------------ >>>>>>>>> acl aci_seniors dstdomain "/blocklist/aci_list/whitelist_seniors" >>>>>>>>> #---------------------------------------------------------#See >>>>>>>>> implimentation in ACI implimentation section >>>>>>>>> >>>>>>>>> #--------------------Assurety Hard_Block-------------- >>>>>>>>> acl aci_hard_block dstdomain "/blocklist/aci_list/hard_block_domains" >>>>>>>>> http_access deny aci_hard_block >>>>>>>>> >>>>>>>>> #--------------------Hard_Block EXE and E.T.C--------------------- >>>>>>>>> #acl mime_block_hard rep_mime_type -i >>>>>>>>> "/blocklist/aci_list/hard_mime_block" >>>>>>>>> #http_reply_access deny mime_block_hard >>>>>>>>> >>>>>>>>> >>>>>>>>> #--General------Streaming Block------------------------------ >>>>>>>>> acl mime_block rep_mime_type -i "/blocklist/aci_list/time_mime_block" >>>>>>>>> >>>>>>>>> #--General Domainlist------------------------------ >>>>>>>>> acl aci_dest dstdomain "/blocklist/aci_list/time_block_domains" >>>>>>>>> >>>>>>>>> #--Seniors MAC list mouting------------------------------ >>>>>>>>> acl aci_mac_seniors arp "/blocklist/aci_list/mac_list_seniors" >>>>>>>>> >>>>>>>>> #--General Timing------------ Normal Days Working hours-------------- >>>>>>>>> acl aci_working_hours time MTWH 10:04-13:04 >>>>>>>>> acl aci_working_hours time MTWH 14:04-18:04 >>>>>>>>> #--General Timing-------------Friday------------------------ >>>>>>>>> acl aci_working_hours time F 10:04-13:04 >>>>>>>>> acl aci_working_hours time F 15:04-18:04 >>>>>>>>> >>>>>>>>> #--General/Seniors-------------Implimentation------------------ >>>>>>>>> http_access allow aci_seniors aci_mac_seniors >>>>>>>>> http_access deny aci_dest aci_working_hours aci_general >>>>>>>>> http_reply_access deny mime_block aci_working_hours aci_general >>>>>>>>> !my_ip >>>>>>>>> >>>>>>>>> #skype deny >>>>>>>>> http_access deny numeric_IPS aci_working_hours >>>>>>>>> http_access deny Skype_UA aci_working_hours >>>>>>>>> http_access deny !validUserAgent aci_working_hours >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> #Error Directory by Ykhan >>>>>>>>> error_directory /usr/share/squid/errors/en-us/ >>>>>>>>> #------------------------TheEnd---------------------- >>>>>>>>> http_access allow aci_lan >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http_access allow manager localhost >>>>>>>>> http_access deny manager >>>>>>>>> http_access allow purge localhost >>>>>>>>> http_access deny purge >>>>>>>>> http_access deny !Safe_ports >>>>>>>>> http_access deny CONNECT !SSL_ports >>>>>>>>> http_access allow localhost >>>>>>>>> http_access deny all >>>>>>>>> icp_access allow localnet >>>>>>>>> icp_access deny all >>>>>>>>> http_port 3128 >>>>>>>>> hierarchy_stoplist cgi-bin ? >>>>>>>>> access_log /var/log/squid/access.log squid >>>>>>>>> refresh_pattern ^ftp: 1440 20% 10080 >>>>>>>>> refresh_pattern ^gopher: 1440 0% 1440 >>>>>>>>> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 >>>>>>>>> refresh_pattern (Release|Package(.gz)*)$ 0 20% 2880 >>>>>>>>> refresh_pattern . 0 20% 4320 >>>>>>>>> acl shoutcast rep_header X-HTTP09-First-Line ^ICY\s[0-9] >>>>>>>>> upgrade_http0.9 deny shoutcast >>>>>>>>> acl apache rep_header Server ^Apache >>>>>>>>> broken_vary_encoding allow apache >>>>>>>>> extension_methods REPORT MERGE MKACTIVITY CHECKOUT >>>>>>>>> hosts_file /etc/hosts >>>>>>>>> coredump_dir /var/spool/squid >>>>>>>>> >>>>>>>>> ##ykhan squid redirection to squidguard >>>>>>>>> >>>>>>>>> #redirect_program /usr/bin/squidGuard >>>>>>>>> #url_rewrite_program /usr/bin/squidGuard >>>>>>>>> #url_rewrite_children 5 >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Apr 23, 2012 at 8:42 PM, Eliezer >>>>>>>>> Croitoru<eliezer@xxxxxxxxxxxx> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 23/04/2012 18:38, Muhammad Yousuf Khan wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> well i have been experiencing slow Internet browsing. not very slow >>>>>>>>>>> but comparatively slower then IPCOP firewall. i can not understand >>>>>>>>>>> how >>>>>>>>>>> come i diagnose the issue. >>>>>>>>>>> i mean. i increase the RAM , i checked the DNS every thing is fine >>>>>>>>>>> but >>>>>>>>>>> my browser stuck at "connecting" ones it start download it do it >>>>>>>>>>> fast >>>>>>>>>>> but then stop for something then start. i am not getting the clear >>>>>>>>>>> picture. can anyone help >>>>>>>>>>> >>>>>>>>>>> i am suing debian 6.0.4 with 2.7 stable squid. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> MYK >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> what is your exact problem? slow downloads? >>>>>>>>>> what is your squid setup?transparent ?regular forward proxy? >>>>>>>>>> what browser are you using? >>>>>>>>>> do you have some squid logs? or squid.conf? >>>>>>>>>> what dns server are you using? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Eliezer >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Eliezer Croitoru >>>>>>>>>> https://www1.ngtech.co.il >>>>>>>>>> IT consulting for Nonprofit organizations >>>>>>>>>> eliezer<at> ngtech.co.il >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Eliezer Croitoru >>>>>> https://www1.ngtech.co.il >>>>>> IT consulting for Nonprofit organizations >>>>>> eliezer<at> ngtech.co.il >>>> >>>> >>>> >>>
