On Thu, Mar 29, 2012 at 4:03 PM, Eliezer Croitoru <eliezer_at_ngtech.co.il> wrote:
> On 29/03/2012 21:05, Carlos Manuel Trepeu Pupo wrote:
>>
>> On Tue, Mar 27, 2012 at 1:23 PM, Eliezer Croitoru<eliezer_at_ngtech.co.il>
>> wrote:
>>>
>>> On 27/03/2012 17:27, Carlos Manuel Trepeu Pupo wrote:
>>>>
>>>>
>>>> On Mon, Mar 26, 2012 at 5:45 PM, Amos Jeffries<squid3_at_treenet.co.nz>
>>>> wrote:
>>>>>
>>>>>
>>>>> On 27.03.2012 10:13, Carlos Manuel Trepeu Pupo wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Mar 24, 2012 at 6:31 PM, Amos Jeffries<squid3_at_treenet.co.nz>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 25/03/2012 7:23 a.m., Carlos Manuel Trepeu Pupo wrote:
>>>>>>>
>>>>>>>> On Thu, Mar 22, 2012 at 10:00 PM, Amos Jeffries wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 23/03/2012 5:42 a.m., Carlos Manuel Trepeu Pupo wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I need to block each user to make just one connection to download
>>>>>>>>>> specific extension files, but I dont know how to tell that can
>>>>>>>>>> make
>>>>>>>>>> just one connection to each file and not just one connection to
>>>>>>>>>> every
>>>>>>>>>> file with this extension.
>>>>>>>>>>
>>>>>>>>>> i.e:
>>>>>>>>>> www.google.com #All connection that required
>>>>>>>>>> www.any.domain.com/my_file.rar #just one connection to that file
>>>>>>>>>> www.other.domain.net/other_file.iso #just connection to this file
>>>>>>>>>> www.other_domain1.com/other_file1.rar #just one connection to that
>>>>>>>>>> file
>>>>>>>>>>
>>>>>>>>>> I hope you understand me and can help me, I have my boss hurrying
>>>>>>>>>> me
>>>>>>>>>> !!!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> There is no easy way to test this in Squid.
>>>>>>>>>
>>>>>>>>> You need an external_acl_type helper which gets given the URI and
>>>>>>>>> decides
>>>>>>>>> whether it is permitted or not. That decision can be made by
>>>>>>>>> querying
>>>>>>>>> Squid
>>>>>>>>> cache manager for the list of active_requests and seeing if the URL
>>>>>>>>> appears
>>>>>>>>> more than once.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hello Amos, following your instructions I make this
>>>>>>>> external_acl_type
>>>>>>>> helper:
>>>>>>>>
>>>>>>>> #!/bin/bash
>>>>>>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep -c
>>>>>>>> "$1"`
>>>>>>>> if [ $result -eq 0 ]
>>>>>>>> then
>>>>>>>> echo 'OK'
>>>>>>>> else
>>>>>>>> echo 'ERR'
>>>>>>>> fi
>>>>>>>>
>>>>>>>> # If I have the same URI then I denied. I make a few test and it
>>>>>>>> work
>>>>>>>> for me. The problem is when I add the rule to the squid. I make
>>>>>>>> this:
>>>>>>>>
>>>>>>>> acl extensions url_regex "/etc/squid3/extensions"
>>>>>>>> external_acl_type one_conn %URI /home/carlos/script
>>>>>>>> acl limit external one_conn
>>>>>>>>
>>>>>>>> # where extensions have:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|mpg|wma|ogg|wmv|asx|asf|deb|rpm|exe|zip|tar|tgz|rar|ppt|doc|tiff|pdf)$
>>>>>>>>
>>>>>>>> http_access deny extensions limit
>>>>>>>>
>>>>>>>>
>>>>>>>> So when I make squid3 -k reconfigure the squid stop working
>>>>>>>>
>>>>>>>> What can be happening ???
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> * The helper needs to be running in a constant loop.
>>>>>>> You can find an example
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://bazaar.launchpad.net/~squid/squid/3.2/view/head:/helpers/url_rewrite/fake/url_fake_rewrite.sh
>>>>>>> although that is re-writer and you do need to keep the OK/ERR for
>>>>>>> external
>>>>>>> ACL.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sorry, this is my first helper, I do not understand the meaning of
>>>>>> running in a constant loop, in the example I see something like I do.
>>>>>> Making some test I found that without this line :
>>>>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep -c
>>>>>> "$1"`
>>>>>> the helper not crash, dont work event too, but do not crash, so i
>>>>>> consider this is in some way the problem.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Squid starts helpers then uses the STDIN channel to pass it a series of
>>>>> requests, reading STDOUt channel for the results. The helper once
>>>>> started
>>>>> is
>>>>> expected to continue until a EOL/close/terminate signal is received on
>>>>> its
>>>>> STDIN.
>>>>>
>>>>> Your helper is exiting without being asked to be Squid after only one
>>>>> request. That is logged by Squid as a "crash".
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> * "eq 0" - there should always be 1 request matching the URL. Which
>>>>>>> is
>>>>>>> the
>>>>>>> request you are testing to see if its>1 or not. You are wanting to
>>>>>>> deny
>>>>>>> for
>>>>>>> the case where there are *2* requests in existence.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> This is true, but the way I saw was: "If the URL do not exist, so
>>>>>> can't be duplicate", I think isn't wrong !!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> It can't not exist. Squid is already servicing the request you are
>>>>> testing
>>>>> about.
>>>>>
>>>>> Like this:
>>>>>
>>>>> receive HTTP request -> (count=1)
>>>>> - test ACL (count=1 -> OK)
>>>>> - done (count=0)
>>>>>
>>>>> receive a HTTP request (count-=1)
>>>>> - test ACL (count=1 -> OK)
>>>>> receive b HTTP request (count=2)
>>>>> - test ACL (count=2 -> ERR)
>>>>> - reject b (count=1)
>>>>> done a (count=0)
>>>>
>>>>
>>>>
>>>> With your explanation and code from Eliezer Croitoru I made this:
>>>>
>>>> #!/bin/bash
>>>>
>>>> while read line; do
>>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep
>>>> -c "$line"`
>>>>
>>>> echo "$line">> /home/carlos/guarda # -> Add this line
>>>> to
>>>> see in a file the $URI I passed to the helper
>>>>
>>>> if [ $result -eq 1 ] # ->
>>>> With your great explain you made me, I change to "1"
>>>> then
>>>> echo 'OK'
>>>> else
>>>> echo 'ERR'
>>>> fi
>>>> done
>>>>
>>>> It's look like it's gonna work, but, here another miss.
>>>> 1- The "echo "$line">> /home/carlos/guarda" do not save anything to
>>>> the
>>>> file.
>>>> 2- When I return 'OK' then in my .conf I can't make a rule like I
>>>> wrote before, I have to make something like this: "http_access deny
>>>> extensions !limit", in the many helps you bring me guys, I learn that
>>>> the name "limit" here its not functional. The deny of "limit" its
>>>> because when there are just one connection I cant block the page.
>>>> 3- With the script just like Eliezer tape it the page with the URL to
>>>> download stay loading infinitely.
>>>>
>>>> So, I have less work, can you help me ??
>>>>
>>>
>>> 1. the first is that "squidclient -h 192.168.19.19 mgr:active_requests"
>>> can
>>> take awhile in some cases.
>>> the first time i tried to run the command it took couple of minutes for
>>> squid to send the list (1 connections).
>>> so your hanging stuff is probably because of this issue.
>>>
>>> 2. why you are writing to a file? if it's for debugging ok.
>>> and what you need to do is ti use the echo $? to get from the grep the
>>> lookup answer first.
>>> so psudo:
>>> -----------
>>> read the uri line. (just notice that there is a possibility for 2 uris on
>>> two different hosts just to notice it._
>>> request from squid the list of active downloads and see if any of the
>>> downloads in the output has a match to the uri in the line before.
>>> in case the uri exists the outpot of "echo $?" (exit code) will produce 0
>>>
>>> case it will find 1 echo OK
>>> case it will find 0 echo ERR
>>>
>>> end
>>> goto read uri...
>>
>>
>> That's correct, I write to a file as debug, and I'm already using this
>> option, thanks
>>
>>>
>>> -----------
>>> the reason you cant add info to the file is because the file is owned by
>>> other user then the one that is executing the script for squid.
>>> so change the file permissions to 666 or change the group and user i
>>> thing
>>> to squid unprivileged user.
>>> the whole thing is a simple while loop with a nice if (echo $? == 1)
>>
>>
>> Great !! That was the error !!! just a simple permissions problem !!!
>> When this things happen me ... grrrrrrrr
>>
>>>
>>>
>>> #!/bin/bash
>>>
>>> while read line; do
>>> #i'm throwing the echo to background in case of slow disc access(dont
>>> really
>>> know how much it will improve)
>>> echo $line>> /home/carlos/guarda&
>>>
>>> # -> Add this line to see in a file the $URI I passed to the helper
>>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep
>>> -c
>>> "$line"|echo $?`
>>>
>>> if [ $result == 1 ]
>>> then
>>> echo 'OK'
>>> echo 'OK'>>/home/carlos/guarda&
>>> else
>>> echo 'ERR'
>>> echo 'ERR'>>/home/carlos/guarda&
>>> fi
>>> done
>>
>>
>> Every page that I tried to download give error, and using the
>> deny_info I see that the acl that deny me it's the related with this
>> external ACL. What could be this ?? I make a lot of test since Tuesday
>> and I don't know what's happening !!! Another thing it this line:
>> result=`squidclient -h 192.168.19.19 mgr:active_requests | grep -c
>> "$line"|echo $?`
>> Why are you using this "|echo $?" at the end of the line and what
>> doing, I know that return the last result code of execute a command or
>> action, but, what you need it here and what do ???
>>
> the instead of storing in memory the grep result i'm storing only the exit
> code of the last command which is the grep.
> if the grep exit when no line with the same uri has been found it will give
> 1 as a result.
> if it found something in the list it will return 0.
I make a few test and always return me 0, no matter what value I enter !!!
> and that is the reason for the OK or ERR on the if condition.
> i will give you a good example that i used on a startup script.
> in order to see if there is an exist process running i would use:
> ps aux |grep -v "grep"|grep -i "some process name"
> so if let say i have a running process named whole_life.sh
> i will get exit code of "0" and i can indicate that it's running and i dont
> need to start it again or i need to kill it.
> so it's very useful to use the exit codes.
>
> i will later try the external_acl stuff on live system to make sure the
> thing is good.
>
> Regards,
> Eliezer
>
>
>
>>>
>>> about the acls.
>>> you can use the follow
>>> http_access deny external_helper_acl
>>>
>>> deny_info http://block_url_site_page/block.html external_helper_acl
>>>
>>> http_access allow loalhost manager
>>> http_access allow loalhost
>>> ...
>>>
>>> this will do the trick for you.
>>
>>
>> Thank I'm already using this.
>>
>>> unless... squidclient is stuck with the output.
>>> and also the echo statements that writes to the file gives error output
>>> that
>>> can cause trouble for that.
>>>
>>> by the way this external acl can limit number of current connections to
>>> more
>>> then just 1 with some wc -l stuff.
>>>
>>> Regards,
>>> ELiezer
>>
>>
>> I think once already, then pass another parameter with the number of
>> current active connections that allow, so make it more flexible.
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> * ensure you have manager requests form localhost not going through
>>>>>>> the
>>>>>>> ACL
>>>>>>> test.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I was making this wrong, the localhost was going through the ACL, but
>>>>>> I just changed !!! The problem persist, What can I do ???
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> which problem?
>>>>>
>>>>>
>>>>> Amos
>>>
>>>
>>>
>>>
>>> --
>>> Eliezer Croitoru
>>> https://www1.ngtech.co.il
>>> IT consulting for Nonprofit organizations
>>> eliezer<at> ngtech.co.il
>
>
>
> --
> Eliezer Croitoru
> https://www1.ngtech.co.il
> IT consulting for Nonprofit organizations
> eliezer <at> ngtech.co.il
Received on Thu Mar 29 2012 - 21:39:14 MDT
This archive was generated by hypermail 2.2.0 : Fri Mar 30 2012 - 12:00:04 MDT