On Thu, Mar 29, 2012 at 4:03 PM, Eliezer Croitoru <eliezer@xxxxxxxxxxxx> wrote: > On 29/03/2012 21:05, Carlos Manuel Trepeu Pupo wrote: >> >> On Tue, Mar 27, 2012 at 1:23 PM, Eliezer Croitoru<eliezer@xxxxxxxxxxxx> >> wrote: >>> >>> On 27/03/2012 17:27, Carlos Manuel Trepeu Pupo wrote: >>>> >>>> >>>> On Mon, Mar 26, 2012 at 5:45 PM, Amos Jeffries<squid3@xxxxxxxxxxxxx> >>>> wrote: >>>>> >>>>> >>>>> On 27.03.2012 10:13, Carlos Manuel Trepeu Pupo wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Mar 24, 2012 at 6:31 PM, Amos Jeffries<squid3@xxxxxxxxxxxxx> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 25/03/2012 7:23 a.m., Carlos Manuel Trepeu Pupo wrote: >>>>>>> >>>>>>>> On Thu, Mar 22, 2012 at 10:00 PM, Amos Jeffries wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 23/03/2012 5:42 a.m., Carlos Manuel Trepeu Pupo wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I need to block each user to make just one connection to download >>>>>>>>>> specific extension files, but I dont know how to tell that can >>>>>>>>>> make >>>>>>>>>> just one connection to each file and not just one connection to >>>>>>>>>> every >>>>>>>>>> file with this extension. >>>>>>>>>> >>>>>>>>>> i.e: >>>>>>>>>> www.google.com #All connection that required >>>>>>>>>> www.any.domain.com/my_file.rar #just one connection to that file >>>>>>>>>> www.other.domain.net/other_file.iso #just connection to this file >>>>>>>>>> www.other_domain1.com/other_file1.rar #just one connection to that >>>>>>>>>> file >>>>>>>>>> >>>>>>>>>> I hope you understand me and can help me, I have my boss hurrying >>>>>>>>>> me >>>>>>>>>> !!! >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> There is no easy way to test this in Squid. >>>>>>>>> >>>>>>>>> You need an external_acl_type helper which gets given the URI and >>>>>>>>> decides >>>>>>>>> whether it is permitted or not. That decision can be made by >>>>>>>>> querying >>>>>>>>> Squid >>>>>>>>> cache manager for the list of active_requests and seeing if the URL >>>>>>>>> appears >>>>>>>>> more than once. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hello Amos, following your instructions I make this >>>>>>>> external_acl_type >>>>>>>> helper: >>>>>>>> >>>>>>>> #!/bin/bash >>>>>>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep -c >>>>>>>> "$1"` >>>>>>>> if [ $result -eq 0 ] >>>>>>>> then >>>>>>>> echo 'OK' >>>>>>>> else >>>>>>>> echo 'ERR' >>>>>>>> fi >>>>>>>> >>>>>>>> # If I have the same URI then I denied. I make a few test and it >>>>>>>> work >>>>>>>> for me. The problem is when I add the rule to the squid. I make >>>>>>>> this: >>>>>>>> >>>>>>>> acl extensions url_regex "/etc/squid3/extensions" >>>>>>>> external_acl_type one_conn %URI /home/carlos/script >>>>>>>> acl limit external one_conn >>>>>>>> >>>>>>>> # where extensions have: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|mpg|wma|ogg|wmv|asx|asf|deb|rpm|exe|zip|tar|tgz|rar|ppt|doc|tiff|pdf)$ >>>>>>>> >>>>>>>> http_access deny extensions limit >>>>>>>> >>>>>>>> >>>>>>>> So when I make squid3 -k reconfigure the squid stop working >>>>>>>> >>>>>>>> What can be happening ??? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> * The helper needs to be running in a constant loop. >>>>>>> You can find an example >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://bazaar.launchpad.net/~squid/squid/3.2/view/head:/helpers/url_rewrite/fake/url_fake_rewrite.sh >>>>>>> although that is re-writer and you do need to keep the OK/ERR for >>>>>>> external >>>>>>> ACL. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Sorry, this is my first helper, I do not understand the meaning of >>>>>> running in a constant loop, in the example I see something like I do. >>>>>> Making some test I found that without this line : >>>>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep -c >>>>>> "$1"` >>>>>> the helper not crash, dont work event too, but do not crash, so i >>>>>> consider this is in some way the problem. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Squid starts helpers then uses the STDIN channel to pass it a series of >>>>> requests, reading STDOUt channel for the results. The helper once >>>>> started >>>>> is >>>>> expected to continue until a EOL/close/terminate signal is received on >>>>> its >>>>> STDIN. >>>>> >>>>> Your helper is exiting without being asked to be Squid after only one >>>>> request. That is logged by Squid as a "crash". >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> * "eq 0" - there should always be 1 request matching the URL. Which >>>>>>> is >>>>>>> the >>>>>>> request you are testing to see if its>1 or not. You are wanting to >>>>>>> deny >>>>>>> for >>>>>>> the case where there are *2* requests in existence. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> This is true, but the way I saw was: "If the URL do not exist, so >>>>>> can't be duplicate", I think isn't wrong !! >>>>> >>>>> >>>>> >>>>> >>>>> It can't not exist. Squid is already servicing the request you are >>>>> testing >>>>> about. >>>>> >>>>> Like this: >>>>> >>>>> receive HTTP request -> (count=1) >>>>> - test ACL (count=1 -> OK) >>>>> - done (count=0) >>>>> >>>>> receive a HTTP request (count-=1) >>>>> - test ACL (count=1 -> OK) >>>>> receive b HTTP request (count=2) >>>>> - test ACL (count=2 -> ERR) >>>>> - reject b (count=1) >>>>> done a (count=0) >>>> >>>> >>>> >>>> With your explanation and code from Eliezer Croitoru I made this: >>>> >>>> #!/bin/bash >>>> >>>> while read line; do >>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep >>>> -c "$line"` >>>> >>>> echo "$line">> /home/carlos/guarda # -> Add this line >>>> to >>>> see in a file the $URI I passed to the helper >>>> >>>> if [ $result -eq 1 ] # -> >>>> With your great explain you made me, I change to "1" >>>> then >>>> echo 'OK' >>>> else >>>> echo 'ERR' >>>> fi >>>> done >>>> >>>> It's look like it's gonna work, but, here another miss. >>>> 1- The "echo "$line">> /home/carlos/guarda" do not save anything to >>>> the >>>> file. >>>> 2- When I return 'OK' then in my .conf I can't make a rule like I >>>> wrote before, I have to make something like this: "http_access deny >>>> extensions !limit", in the many helps you bring me guys, I learn that >>>> the name "limit" here its not functional. The deny of "limit" its >>>> because when there are just one connection I cant block the page. >>>> 3- With the script just like Eliezer tape it the page with the URL to >>>> download stay loading infinitely. >>>> >>>> So, I have less work, can you help me ?? >>>> >>> >>> 1. the first is that "squidclient -h 192.168.19.19 mgr:active_requests" >>> can >>> take awhile in some cases. >>> the first time i tried to run the command it took couple of minutes for >>> squid to send the list (1 connections). >>> so your hanging stuff is probably because of this issue. >>> >>> 2. why you are writing to a file? if it's for debugging ok. >>> and what you need to do is ti use the echo $? to get from the grep the >>> lookup answer first. >>> so psudo: >>> ----------- >>> read the uri line. (just notice that there is a possibility for 2 uris on >>> two different hosts just to notice it._ >>> request from squid the list of active downloads and see if any of the >>> downloads in the output has a match to the uri in the line before. >>> in case the uri exists the outpot of "echo $?" (exit code) will produce 0 >>> >>> case it will find 1 echo OK >>> case it will find 0 echo ERR >>> >>> end >>> goto read uri... >> >> >> That's correct, I write to a file as debug, and I'm already using this >> option, thanks >> >>> >>> ----------- >>> the reason you cant add info to the file is because the file is owned by >>> other user then the one that is executing the script for squid. >>> so change the file permissions to 666 or change the group and user i >>> thing >>> to squid unprivileged user. >>> the whole thing is a simple while loop with a nice if (echo $? == 1) >> >> >> Great !! That was the error !!! just a simple permissions problem !!! >> When this things happen me ... grrrrrrrr >> >>> >>> >>> #!/bin/bash >>> >>> while read line; do >>> #i'm throwing the echo to background in case of slow disc access(dont >>> really >>> know how much it will improve) >>> echo $line>> /home/carlos/guarda& >>> >>> # -> Add this line to see in a file the $URI I passed to the helper >>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep >>> -c >>> "$line"|echo $?` >>> >>> if [ $result == 1 ] >>> then >>> echo 'OK' >>> echo 'OK'>>/home/carlos/guarda& >>> else >>> echo 'ERR' >>> echo 'ERR'>>/home/carlos/guarda& >>> fi >>> done >> >> >> Every page that I tried to download give error, and using the >> deny_info I see that the acl that deny me it's the related with this >> external ACL. What could be this ?? I make a lot of test since Tuesday >> and I don't know what's happening !!! Another thing it this line: >> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep -c >> "$line"|echo $?` >> Why are you using this "|echo $?" at the end of the line and what >> doing, I know that return the last result code of execute a command or >> action, but, what you need it here and what do ??? >> > the instead of storing in memory the grep result i'm storing only the exit > code of the last command which is the grep. > if the grep exit when no line with the same uri has been found it will give > 1 as a result. > if it found something in the list it will return 0. I make a few test and always return me 0, no matter what value I enter !!! > and that is the reason for the OK or ERR on the if condition. > i will give you a good example that i used on a startup script. > in order to see if there is an exist process running i would use: > ps aux |grep -v "grep"|grep -i "some process name" > so if let say i have a running process named whole_life.sh > i will get exit code of "0" and i can indicate that it's running and i dont > need to start it again or i need to kill it. > so it's very useful to use the exit codes. > > i will later try the external_acl stuff on live system to make sure the > thing is good. > > Regards, > Eliezer > > > >>> >>> about the acls. >>> you can use the follow >>> http_access deny external_helper_acl >>> >>> deny_info http://block_url_site_page/block.html external_helper_acl >>> >>> http_access allow loalhost manager >>> http_access allow loalhost >>> ... >>> >>> this will do the trick for you. >> >> >> Thank I'm already using this. >> >>> unless... squidclient is stuck with the output. >>> and also the echo statements that writes to the file gives error output >>> that >>> can cause trouble for that. >>> >>> by the way this external acl can limit number of current connections to >>> more >>> then just 1 with some wc -l stuff. >>> >>> Regards, >>> ELiezer >> >> >> I think once already, then pass another parameter with the number of >> current active connections that allow, so make it more flexible. >> >>> >>> >>> >>>> >>>> >>>>> >>>>> >>>>>> >>>>>>> >>>>>>> * ensure you have manager requests form localhost not going through >>>>>>> the >>>>>>> ACL >>>>>>> test. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I was making this wrong, the localhost was going through the ACL, but >>>>>> I just changed !!! The problem persist, What can I do ??? >>>>> >>>>> >>>>> >>>>> >>>>> which problem? >>>>> >>>>> >>>>> Amos >>> >>> >>> >>> >>> -- >>> Eliezer Croitoru >>> https://www1.ngtech.co.il >>> IT consulting for Nonprofit organizations >>> eliezer<at> ngtech.co.il > > > > -- > Eliezer Croitoru > https://www1.ngtech.co.il > IT consulting for Nonprofit organizations > eliezer <at> ngtech.co.il
