Re: Res: [squid-users] squid 3.2.0.5 smp scaling issues

On 26/04/11 11:31, david_at_lang.hm wrote:
> On Mon, 25 Apr 2011, david_at_lang.hm wrote:
>
>> On Mon, 25 Apr 2011, Alex Rousskov wrote:
>>
>>> On 04/14/2011 09:06 PM, david_at_lang.hm wrote:
>>>
>>>> In addition, there seems to be some sort of locking betwen the multiple
>>>> worker processes in 3.2 when checking the ACLs
>>>
>>> There are pretty much no locks in the current official SMP code. This
>>> will change as we start adding shared caches in a week or so, but even
>>> then the ACLs will remain lock-free. There could be some internal
>>> locking in the 3rd-party libraries used by ACLs (regex and such), but I
>>> do not know much about them.
>>
>> what are the 3rd party libraries that I would be using?
>
> one thought I had is that this could be locking on name lookups. how
> hard would it be to create a quick patch that would bypass the name
> lookups entirely and only do the lookups by IP.

If your ACLs are doing name lookups on data which is an IP ...

  ie "src machine-x.example.com" - they should only be slowing the
startup/reconfigure process as the IPs are looked up to add to the ACL
and used for the entire run-time (quite bad to do if your machines are
dynamic).

  ie "dst example.com" - they are looked up once per ACL test in 3.0-
and once per different *_access list in 3.1+

  The other domain/IP tests are done without any lookups since data is
already available from the client request.

>
> if that regains the speed and/or scalability it would point fingers
> fairly conclusively at the DNS components.

There may be other factors than locking coming into effect of course.
Packet loss under traffic load, recursive resolver load etc.

As for libraries, you have only mentioned regex ACLs, so whatever regex
library your OS provides. If you have the Squid bundled GnuRegex there
is no locking at all.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.12
   Beta testers wanted for 3.2.0.7 and 3.1.12.1