Re: [squid-users] squid_session problem

On 10.07.2012 15:12, Jack Black wrote:
> On Mon, Jul 9, 2012 at 7:48 PM, Amos Jeffries wrote:
>> On 10.07.2012 13:18, Jack Black wrote:
>>>
>>> Hi.
>>>
>>> Has anyone successfully used squid, and the squid_session helper in
>>> order to force users of the proxy server to see a webpage (be
>>> redirected to it) at the beginning of each session?
>>
>>
>> Yes, many.
>>
>>
>>>
>>> After spending weeks trying to get this to work, I was finally
>>> successful using squid version 3.1.10 on CentOS. Unfortunately, I
>>> need
>>> it to work on Ubuntu Server 12.04 with squid version 3.1.19
>>> instead,
>>> and doing exactly the same thing as I did in CentOS, this fails to
>>> work on the Ubuntu Server, and my /var/log/squid3/cache.log has a
>>> line
>>> similar to:
>>>
>>>> externalAclLookup: 'session' queue overload (ch=0x....)
>>
>>
>> HINT: "queue overload" - you are getting more requests per second
>> than the
>> helper can reply to. Even with TTL > 0.
>>
>> I'm a bit suspicious that with children=1 the queue is only 2
>> entries long
>> (not good). Since it is based on number of helpers, and seems not to
>> account
>> for concurrency. The log message could be due to that, but that
>> would not
>> allow requests past the splash page, quite the opposite in fact.
>
> How can you tell the queue is only 2 entries long?

The queue is 2x n_children, or 2x1 in length, with 200x n_children
slots of concurrency to use up before the queue starts filling. It
should not matter if the helper is fast enough, but explains that queue
overload message will appear if you have 202 lookups waiting for a
response.

> Am I missing
> something? My main focus during the weeks I spent getting this to
> work
> was getting squid to talk to Cisco using WCCP. I am still far from
> understanding exactly how this helper works, and am no squid expert.

The helper maintains a database of previously seen strings/sessions. In
your case a database of previously seen IP addresses ("%SRC").

  When there is no existing entry in the session database, one is
created immediately and ERR/no-match is returned. http_access results in
a no-match and !session makes that a pass, so the deny is done and
splash page is displayed.
  When a lookup is done the database is checked, any existing entry
results in O/match and the DB record is updated to keep it another 900
seconds (-t 900). http_access results in a match and !session makes that
a fail, so the deny is not done.
  Combining these, you can see that the first request gets the splash
page, and following ones get the requested content.

Using the %SRC, means any packets arriving from that IP can do the
above and you have almost no control over whether an actual user or some
blind automated software is getting the splash page.
  It is extremely easy for automated bits (I listed a few earlier) to
end up with the splash p[age and users not even noticing the sessions.
Which is what you described happening.

> I
> found the lines below for configuring squid_session in squid.conf
> online (I didn't write them), and only slightly changed them to work
> for me. The only instructions I can find online for how the
> squid_session helper works is an exact copy of the man page for
> squid_session, which only has one example and not much explanation
> for
> what the different values mean on the line that starts with
> "external_acl_type" (the man page has no mention of ttl, negative
> ttl,
> children, etc...).

Those are all options for the external_acl_type directive. Not the
session helper itself. External ACL is an interface used by Squid to run
custom scripts that do complex access control things.
  It is documented at
http://www.squid-cache.org/Doc/config/external_acl_type/

There is a bit more documentation about the session helper and how it
does splash pages at
http://wiki.squid-cache.org/ConfigExamples/Portal/Splash

>
> Oh and while my final goal is to get this working with WCCP, right
> now
> I'm testing without it so as to keep things simple.
>
>>
>>
>>>
>>> for every http request my client sends (so a lot of those lines).
>>> The
>>> client is forwarded through the squid proxy directly to the page
>>> they
>>> request every time, and the splash page is always ignored. Here are
>>> the relevant lines from squid.conf:
>>>
>>>> external_acl_type session ttl=300 negative_ttl=0 children=1
>>>> concurrency=200 %SRC /usr/lib/squid3/squid_session -t 900
>>>> acl session external session
>>>> http_access deny !session
>>>> deny_info http://example.com session
>>>
>>>
>>> Does anyone know the problem? Am I doing something wrong?
>>>
>>> Tal
>>
>>
>> Splash page is only at the beginning of a browsing "session". If
>> their
>> requests are less then 900 seconds apart the existing session is
>> extended
>> another 900 seconds from that time.
>>
>> * you are making a session based on any HTTP request made by that
>> IP
>> address, so *all* software operating on the client machine can
>> trigger an
>> HTTP request which extends the session or creates a new one.
>
> I am aware of this, and having restarted squid many times in order to
> reset the session counters, while surfing the web on the client at
> the
> exact time squid starts, I have not seen the splash screen once (in
> Ubuntu Server), making me fairly confident it's not working. When
> this
> is done on CentOS and everything works, the splash screen is easy to
> run into with exactly the same client.

Ah. If you run the session helper without a disk-backed database it
keeps everything in RAM. A simple reconfigure or log rotation is enough
to restart the helper and wipe that RAM database clean.

If you are using disk-backed, it will persist across even a full
stop-start cycle of Squid. Use short -t values to make sessions expire
quickly while testing that setup.

If something like the automated tools are getting the splash page the
only sign you will have of it is where the first line of access.log
comes from. Session is created immediately and the only sign in Squid is
the initial TCP_DENIED followed by acceptance.

You might want to use active mode to ensure no session is created until
after your special splash page URL has been requested by someone. Your
access.log will then show clearly if anything is doing that since the
http://example.com URL will show up whenever that happens.

>
>> * it is common to have multiple software updaters active on the PC,
>> which
>> use HTTP to fetch their updates. Some of those may be running first
>> and
>> getting the splash page so the user never sees it.
>
> I am aware, and that is why I test multiple times, restarting squid
> each time, in order to see the splash page even once.
>
>> * it is common for browsers these days to do some form of CORS
>> lookup
>> before starting a page fetch. If your splash page is what comes back
>> to that
>> CORS request the naive security checks will fail and the website
>> they wanted
>> will appear without anything showing up.
>
> I was not aware of this - maybe that is what's happening here -
> though
> I'm not sure why it would trigger a session with Ubuntu, but not
> CentOS as the squid proxy server. I'll have to do some reading on
> this. Thanks for the hint.
>
>> Any one of those, or none of them could be the problem.
>>
>> Start with trying 3.1.19 on CentOS. Building it yourself from the
>> Ubuntu
>> source package would be a good test to see if it is a change in
>> Squid over
>> those 14 releases or something in the system.
>>
>> Amos
>>
>
> I've tried compiling newer and older versions of squid on the Ubuntu
> Server, and ran into the same problem. I will give it another try
> though - maybe I missed something.
>
> Thanks again - always helps to have fresh eyes on a problem like
> this.
>
> Tal

Amos
Received on Tue Jul 10 2012 - 03:46:27 MDT