Re: [squid-users] Can url_rewrite_program determine the referer?

On Sun, Oct 16, 2011 at 10:03 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On 17/10/11 15:51, dustfinger x wrote:
>>
>> On Sun, Oct 9, 2011 at 4:43 PM, Amos Jeffries<squid3_at_treenet.co.nz>
>>  wrote:
>>>
>>> On Sun, 9 Oct 2011 16:07:00 -0600, dustfinger x wrote:
>>>>
>>>> On Sun, Oct 9, 2011 at 12:19 PM, Diego Woitasen
>>>> <diego_at_woitasen.com.ar>  wrote:
>>>>>
>>>>> On Sun, Oct 9, 2011 at 11:10 AM, dustfinger x
>>>>> <dustfinger_at_muddymukluk.com>  wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> The system is using Squid version 3.1.8.
>>>>>>
>>>>>> I have configured squid to use a url_rewrite_program that redirects
>>>>>> users to the company portal sites under certain circumstances. The
>>>>>> problem is that the portal sites references external content and the
>>>>>> external content URI's are also being re_written. Is there any way for
>>>>>> me to determine if a particular uri has a referer? It would be idea if
>>>>>> I could determine the referer's domain name, but even just knowing if
>>>>>> the uri request has a referer would be helpful.
>>>>>>
>>>>>> Sincerely,
>>>>>>
>>>>>> dustfinger.
>>>>>>
>>>>>
>>>>> Use "acl referer_regex" and url_rewriter_access. For example:
>>>>>
>>>>> acl intranet_ref referer_regex *.intranet.com.*
>>>>> url_rewriter_access deny intranet_ref
>>>>> url_rewriter_access allow all
>>>>>
>>>>> Regards,
>>>>>  Diego
>>>>>
>>>>>
>>>>> --
>>>>> Diego Woitasen
>>>>>
>>>>
>>>> Hi Diego,
>>>>
>>>> Thank you very much for your response. The challenge that I face is
>>>> that there will probably be a lot of referred to domains and these may
>>>> potentially change over time. If it is possible to somehow determine
>>>> the referrer from the url_rewrite_program, then that would be idea.
>>>> Your solution is not totally out of the question, but it would be a
>>>> maintenance issue for me.
>>>>
>>>> I suspect that what I want to do is simply not supported, but I did
>>>> read one user's post that he was able to pass the referrer to the
>>>> redirector using user variables. The poster did not detail how he went
>>>> about this though.
>>>
>>> Sounds way too complex. You can use external ACL in url_rewrite_access to
>>> make the ACL checks real-time based on arbitrary data source. Usage the
>>> same
>>> as in http_access.
>>>
>>> Amos
>>>
>>
>> Hi,
>>
>> Correct me if I am wrong, but in your suggested solution I would still
>> have to know in advance all of the domain names that I wanted to
>> redirect, or all of the domain names that I do not want to redirect.
>> Is it possible to use your suggested solution in the following
>> scenario:
>>
>> Suppose that I have in a database a list of URI's that I would like to
>> allow access to. Consider one of these URI's and let's referrer to
>> that URI as URI_A. It turns out that URI_A contains content that is
>> hosted by a domain not contained in our database of URI's that we
>> would like to allow access to. Let's refer to the referred-to URI as
>> URI_UNLISTED. Now when a user requests content from URI_A, any of the
>> content that is referred to by URI_A, but is hosted by URI_UNLISTED,
>> is redirected. The result is that none of the reffered to content will
>> be returned to the client and that is not the behavior that I am
>> looking for.
>
> If I understand that right you want:
>  * unknown page 'URI_A' which is pointing *to* one of your acceptable URLs
> to be automatically accepted.
>  * AND you want all other references that page makes (URI_UNLISTED) to be
> also automatically accepted.
>
> Assuming that is correct, two major problems:
>  1) what if the URI_A refers to a URI_UNLISTED before it refers to the
> acceptable URL?
>  2) what if malicious person adds reference to one of your acceptable URLs
> to a page you actually want blocked?
>
> I hope I misunderstood your above paragraph. The one below is clearer and
> seems to describe a safer set of requirements.
>
>>
>> This is what I am looking for. If a client makes a direct request to
>> URI_UNLISTED, then I would like to redirect that request by rewriting
>> the URI. If the client makes a request to URI_A, and URI_A refers to
>> URI_UNLISTED, then I would like all of the content to be accessible,
>> with no URI rewriting. That is, the request to URI_UNLISTED is
>> accepted since it is being referred to by a URI that is in our
>> database.
>>
>> I know that if I could gain access to the referrer in the
>> url_rewrite_program, then I could achieve this behavior.
>>
>> Does anyone know how I could achieve the behavior that I have described.
>
>
> You don't need a URL-change helper. You need an access control helper.
>
> This is how I would code up the config and script to meet your scenario
> requirements:
>
>  # external ACL helper. To determine if this URL request is acceptable.
>  # may be a read-only lookup, or may add thing to the database live.
>  # whatever you desire as side-effects, it could do.
>
>  external_acl_type urlTest %URI %{Referer}>h /path/to/script
>
>  acl urlIsOkay external urlTest
>
>  # redirect to this URL if the requested URL is bad.
>  deny_info http://example.com/badurl.html urlIsOkay
>
>  # deny the bad URL requests.
>  # NOTE: using allow here will not trigger the redirect above.
>  http_access deny !urlIsOkay
>
>  # the regular
>  http_access allow localnet
>  http_access deny all
>
>
> You risk malicious persons adding forged Referer: headers to their requests
> in order to get past your access controls. This is a standing risk with
> depending on the Referer in any security system you need to be aware of.
>
> see http://www.squid-cache.org/Doc/config/external_acl_type for details on
> the external ACL directive and its parameters. What I wrote above was an
> example and you will need to tune things further.
>
> Amos
> --
> Please be using
>  Current Stable Squid 2.7.STABLE9 or 3.1.16
>  Beta testers wanted for 3.2.0.13
>

Hi Amos,

The statement below is not what I had intended, but after rereading my
scenario I can see that my wording was not that clear.
> If I understand that right you want:
> * unknown page 'URI_A' which is pointing *to* one of your acceptable URLs
> to be automatically accepted.
> * AND you want all other references that page makes (URI_UNLISTED) to be
> also automatically accepted.

Your rewording of my requirements is correct and your suggested
solution seems like it is what I am looking for. I am going to try and
implement that in the next couple of days, thank you very much for
your help on this.

Sincerely,

dustfinger.
Received on Tue Oct 18 2011 - 03:59:30 MDT