Re: [squid-users] squid_session problem

I just tried the same squid configuration on Ubuntu Mini remix 11.04
x64, and Ubuntu Desktop 12.04 x64 live (I didn't even bother
installing it). They both work perfectly, just like CentOS. Whatever
the problem is, it appears to be specific to Ubuntu 12.04 Server
edition for some reason, which just happens to be the exact OS my
production server (the one that needs to run squid) is running.
There's got to be something that I'm missing...

Tal

On Wed, Jul 11, 2012 at 3:30 PM, Jack Black <secretagent101_at_gmail.com> wrote:
> I have made a video that proves that this is not just a case of some
> other program getting the splash page, but it was too big for the
> mailing lists, so I uploaded it here:
>
> http://dxgameunit.webs.com/squid.ogv
>
> What it basically shows is:
>
> a) squid running in a CentOS VM, and access.log messages are being
> displayed on screen while I surf the net on the client. As soon as
> squid starts up and registers with the router using WCCP (I do use
> WCCP in this case), the first page the client opens after that point
> creates a new session, causing it to be redirected to a page of my
> choosing, as it should. This also causes a TCP_DENIED entry in
> access.log, as expected. All subsequent pages go through fine until
> the session expires.
>
> b) squid running in an Ubuntu Server VM, and access.log messages
> displayed on screen as well, while I continue to surf the net on the
> client. As soon as squid starts receiving traffic, it simply forwards
> all traffic on to the web. At no point is there a TCP_DENIED response
> recorded in access.log like there is with CentOS, indicating that
> redirection has taken place.
>
> While reading up on external_acl_type and squid_session is slowly
> helping me understand how they work, I don't think the problem is with
> the syntax. I have already tried copying the squid.conf file from the
> working CentOS to the non-working Ubuntu Server, with no improvement.
> I have also just compiled squid 3.2.0.18 (the latest beta) on Ubuntu
> Server, and used the active mode configuration below:
>
> external_acl_type session concurrency=100 ttl=3 %SRC
> /usr/local/squid/libexec/ext_session_acl -a -T 60
> acl session_login external session LOGIN
> external_acl_type session_active_def concurrency=100 ttl=3 %SRC
> /usr/local/squid/libexec/ext_session_acl -a -T 60
> acl session_is_active external session_active_def
> acl clicked_login_url url_regex -i a-url-that-must-match$
> http_access allow clicked_login_url session_login
> http_access deny !session_is_active
> deny_info 511:/var/www/index.html session_is_active
>
> The result was exactly the same as before - a lot of lines that read:
>
> externalAclLookup 'session_active_def' queue overload
>
> in the cache.log file, and no redirection to the splash page on the
> client. I'm thinking that something deep in the OS (maybe some /proc
> setting?) is causing the queue overload. You mentioned earlier that:
>
>>HINT: "queue overload" - you are getting more requests per second than the
>>helper can reply to. Even with TTL > 0.
>
> but I'm only making a few requests per second for a few seconds from a
> single client, on a quad core machine (the VM has access to all 4
> cores, as well as 2GB of RAM). Surely it should be able to handle
> that. I have also tried the same Ubuntu Server and squid setup on an
> actual, physical machine with a 2.5GHz Xeon CPU, and 2GB of RAM. The
> problem it ran into is exactly the same as the one in the video.
>
> Any more suggestions would be highly appreciated.
>
> Tal
>
> On Mon, Jul 9, 2012 at 9:46 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> On 10.07.2012 15:12, Jack Black wrote:
>>
>>> On Mon, Jul 9, 2012 at 7:48 PM, Amos Jeffries wrote:
>>>>
>>>> On 10.07.2012 13:18, Jack Black wrote:
>>>>>
>>>>>
>>>>> Hi.
>>>>>
>>>>> Has anyone successfully used squid, and the squid_session helper in
>>>>> order to force users of the proxy server to see a webpage (be
>>>>> redirected to it) at the beginning of each session?
>>>>
>>>>
>>>>
>>>> Yes, many.
>>>>
>>>>
>>>>>
>>>>> After spending weeks trying to get this to work, I was finally
>>>>> successful using squid version 3.1.10 on CentOS. Unfortunately, I need
>>>>> it to work on Ubuntu Server 12.04 with squid version 3.1.19 instead,
>>>>> and doing exactly the same thing as I did in CentOS, this fails to
>>>>> work on the Ubuntu Server, and my /var/log/squid3/cache.log has a line
>>>>> similar to:
>>>>>
>>>>>> externalAclLookup: 'session' queue overload (ch=0x....)
>>>>
>>>>
>>>>
>>>> HINT: "queue overload" - you are getting more requests per second than
>>>> the
>>>> helper can reply to. Even with TTL > 0.
>>>>
>>>> I'm a bit suspicious that with children=1 the queue is only 2 entries
>>>> long
>>>> (not good). Since it is based on number of helpers, and seems not to
>>>> account
>>>> for concurrency. The log message could be due to that, but that would not
>>>> allow requests past the splash page, quite the opposite in fact.
>>>
>>>
>>> How can you tell the queue is only 2 entries long?
>>
>>
>> The queue is 2x n_children, or 2x1 in length, with 200x n_children slots of
>> concurrency to use up before the queue starts filling. It should not matter
>> if the helper is fast enough, but explains that queue overload message will
>> appear if you have 202 lookups waiting for a response.
>>
>>
>>
>>> Am I missing
>>> something? My main focus during the weeks I spent getting this to work
>>> was getting squid to talk to Cisco using WCCP. I am still far from
>>> understanding exactly how this helper works, and am no squid expert.
>>
>>
>> The helper maintains a database of previously seen strings/sessions. In your
>> case a database of previously seen IP addresses ("%SRC").
>>
>>
>> When there is no existing entry in the session database, one is created
>> immediately and ERR/no-match is returned. http_access results in a no-match
>> and !session makes that a pass, so the deny is done and splash page is
>> displayed.
>> When a lookup is done the database is checked, any existing entry results
>> in O/match and the DB record is updated to keep it another 900 seconds (-t
>> 900). http_access results in a match and !session makes that a fail, so the
>> deny is not done.
>> Combining these, you can see that the first request gets the splash page,
>> and following ones get the requested content.
>>
>> Using the %SRC, means any packets arriving from that IP can do the above and
>> you have almost no control over whether an actual user or some blind
>> automated software is getting the splash page.
>> It is extremely easy for automated bits (I listed a few earlier) to end up
>> with the splash p[age and users not even noticing the sessions. Which is
>> what you described happening.
>>
>>
>>
>>> I
>>> found the lines below for configuring squid_session in squid.conf
>>> online (I didn't write them), and only slightly changed them to work
>>> for me. The only instructions I can find online for how the
>>> squid_session helper works is an exact copy of the man page for
>>> squid_session, which only has one example and not much explanation for
>>> what the different values mean on the line that starts with
>>> "external_acl_type" (the man page has no mention of ttl, negative ttl,
>>> children, etc...).
>>
>>
>> Those are all options for the external_acl_type directive. Not the session
>> helper itself. External ACL is an interface used by Squid to run custom
>> scripts that do complex access control things.
>> It is documented at
>> http://www.squid-cache.org/Doc/config/external_acl_type/
>>
>>
>> There is a bit more documentation about the session helper and how it does
>> splash pages at
>> http://wiki.squid-cache.org/ConfigExamples/Portal/Splash
>>
>>
>>
>>>
>>> Oh and while my final goal is to get this working with WCCP, right now
>>> I'm testing without it so as to keep things simple.
>>>
>>>>
>>>>
>>>>>
>>>>> for every http request my client sends (so a lot of those lines). The
>>>>> client is forwarded through the squid proxy directly to the page they
>>>>> request every time, and the splash page is always ignored. Here are
>>>>> the relevant lines from squid.conf:
>>>>>
>>>>>> external_acl_type session ttl=300 negative_ttl=0 children=1
>>>>>> concurrency=200 %SRC /usr/lib/squid3/squid_session -t 900
>>>>>> acl session external session
>>>>>> http_access deny !session
>>>>>> deny_info http://example.com session
>>>>>
>>>>>
>>>>>
>>>>> Does anyone know the problem? Am I doing something wrong?
>>>>>
>>>>> Tal
>>>>
>>>>
>>>>
>>>> Splash page is only at the beginning of a browsing "session". If their
>>>> requests are less then 900 seconds apart the existing session is extended
>>>> another 900 seconds from that time.
>>>>
>>>> * you are making a session based on any HTTP request made by that IP
>>>> address, so *all* software operating on the client machine can trigger an
>>>> HTTP request which extends the session or creates a new one.
>>>
>>>
>>> I am aware of this, and having restarted squid many times in order to
>>> reset the session counters, while surfing the web on the client at the
>>> exact time squid starts, I have not seen the splash screen once (in
>>> Ubuntu Server), making me fairly confident it's not working. When this
>>> is done on CentOS and everything works, the splash screen is easy to
>>> run into with exactly the same client.
>>
>>
>>
>> Ah. If you run the session helper without a disk-backed database it keeps
>> everything in RAM. A simple reconfigure or log rotation is enough to restart
>> the helper and wipe that RAM database clean.
>>
>> If you are using disk-backed, it will persist across even a full stop-start
>> cycle of Squid. Use short -t values to make sessions expire quickly while
>> testing that setup.
>>
>> If something like the automated tools are getting the splash page the only
>> sign you will have of it is where the first line of access.log comes from.
>> Session is created immediately and the only sign in Squid is the initial
>> TCP_DENIED followed by acceptance.
>>
>>
>> You might want to use active mode to ensure no session is created until
>> after your special splash page URL has been requested by someone. Your
>> access.log will then show clearly if anything is doing that since the
>> http://example.com URL will show up whenever that happens.
>>
>>
>>
>>>
>>>> * it is common to have multiple software updaters active on the PC,
>>>> which
>>>> use HTTP to fetch their updates. Some of those may be running first and
>>>> getting the splash page so the user never sees it.
>>>
>>>
>>> I am aware, and that is why I test multiple times, restarting squid
>>> each time, in order to see the splash page even once.
>>>
>>>> * it is common for browsers these days to do some form of CORS lookup
>>>> before starting a page fetch. If your splash page is what comes back to
>>>> that
>>>> CORS request the naive security checks will fail and the website they
>>>> wanted
>>>> will appear without anything showing up.
>>>
>>>
>>> I was not aware of this - maybe that is what's happening here - though
>>> I'm not sure why it would trigger a session with Ubuntu, but not
>>> CentOS as the squid proxy server. I'll have to do some reading on
>>> this. Thanks for the hint.
>>>
>>>> Any one of those, or none of them could be the problem.
>>>>
>>>> Start with trying 3.1.19 on CentOS. Building it yourself from the Ubuntu
>>>> source package would be a good test to see if it is a change in Squid
>>>> over
>>>> those 14 releases or something in the system.
>>>>
>>>> Amos
>>>>
>>>
>>> I've tried compiling newer and older versions of squid on the Ubuntu
>>> Server, and ran into the same problem. I will give it another try
>>> though - maybe I missed something.
>>>
>>> Thanks again - always helps to have fresh eyes on a problem like this.
>>>
>>> Tal
>>
>>
>>
>>
>> Amos
Received on Thu Jul 12 2012 - 00:37:56 MDT