Re: [squid-users] Too many queued ntlmauthenticator requests and squid failed

On 21/10/10 13:07, Tharanga Abeyseela (RGA) wrote:
> Hi Guys,
>
> This is my http_access sequence, but iam using redirector_access deny at the top, can it cause too may ntlmauthenticator requests ?
>

redirector_access do not get checked until long after http_access.

> Please have a look and let me know your thoughts,
>
>
> auth_param ntlm program /usr/local/squid/libexec/ntlm_auth --helper-protocol=squid-2.5-ntlmssp

Um, that .../squid/libexec/... path indicates the half-broken Squid
LanMan helper is being used. But the command line parameter is for the
Samba NTLM authenticator.

I recommend locating the actual Samba helper and change to using it instead.

> auth_param ntlm children 25
> auth_param basic program /usr/local/squid/libexec/ntlm_auth --helper-protocol=squid-2.5-basic

The squid ntlm_auth helper does not understand Basic protocol syntax. If
this fallback is being used this could be increasing your overload problem.

<snip>
>
> external_acl_type nt_group ttl=0 children=25 %LOGIN /usr/local/squid/libexec/wbinfo_group.pl
> acl AuthorizedUsers proxy_auth REQUIRED
> acl NoRedirect external nt_group ProxyUnrestAccess
> acl youtube_domains dstdomain .youtube.com
> acl utubeallow external nt_group ProxyYouTubeAccess
> acl restrictedusers external nt_group proxyaccessfull
>
>
>
> redirecotry_access deny Noredirect

> http_access allow utubeallow

First thing your config does on receiving a new request is challenge for
auth. (%LOGIN requires auth to be done).

You seem to be missing these front-line security protections:
   http_access deny !Safe_ports
   http_access deny CONNECT !SSL_ports

they are designed to protect your Squid from overload by malicious proxy
scans and internal LAN infections. Place the *first* in your http_access
lines and adjust the content of SSL_Ports and Safe_ports to contain any
unusual ports you need opened.

Amos

> http_access deny youtube_domains
> http_access allow AuthorizedUsers restrictedusers
> http_access allow localnet AuthorizedUsers restrictedusers
> http_access deny all
>
>
> Thanks for your time.
>
> Tharanga
>
>
>
>
> -----Original Message-----
> From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
> Sent: Tuesday, 19 October 2010 10:15 AM
> To: Tharanga Abeyseela (RGA)
> Cc: squid-users_at_squid-cache.org
> Subject: RE: [squid-users] Too many queued ntlmauthenticator requests and squid failed
>
> On Mon, 18 Oct 2010 22:54:42 +0000, "Tharanga Abeyseela (RGA)" wrote:
>> Hi,
>>
>> I tried to start squid in the offpeak time there were only 35 users, but
>> still its says Too many queued ntlmauthenticator requests, and terminate
>> the squid process.
>>
>> My domain is connected to several other domains (trusted domains) can it
>> cause this issue ?
>>
>>
>> Following are the error messages on log files.
>>
>> Squid Parent: child process 15206 exited with status 1
>> Oct 18 19:47:29 rapsydprox2 squid[15204]: Squid Parent: child process
>> 20806 exited due to signal 15
>>
>> 2010/10/18 19:47:10, 2] winbindd/winbindd.c:872(remove_client)
>> final write to client failed: Broken pipe
>> [2010/10/18 19:47:10, 0]
>> winbindd/winbindd.c:954(winbindd_listen_fde_handler)
>> winbindd: Exceeding 200 client connections, no idle connection found
>
> Ouch. So your problem then is that your Squid receives greater than *200*
> concurrent *new* TCP connections (per second or so) with only 35 users.
> This is about double the maximum number of concurrent connections modern
> web browser make to proxies (4-6).
> * Make sure that your squid.conf has at least
> client_persistent_connections and server_persistent_connections turned on.
> It might reduce this a little bit.
> * check your squid.conf http_access listing order. Make as many
> rejections as possible before testing for auth. This will usually have a
> chain reaction down to increased average service times.
>
> Note that squid-3.0 does not have "connection pinning" which is required
> to connect to anything on the other side of Squid with NTLM. Connections
> *to* Squid with NTLM only required persistent connections.
>
> I really strongly urge you move to 3.1.8. We have done a lot of work to
> make connections more persistent since 3.0 was released.
>
>
>> [2010/10/18 19:47:21, 0]
>> winbindd/winbindd.c:954(winbindd_listen_fde_handler)
>> winbindd: Exceeding 200 client connections, no idle connection found
>> [2010/10/18 19:47:21, 2] winbindd/winbindd.c:872(remove_client)
>> final write to client failed: Broken pipe
>>
>>
>> 5 Oct 18 16:58:37 rapsydprox2 (squid): Too many queued
> ntlmauthenticator
>> requests
>> 976 Oct 18 17:00:23 rapsydprox2 (ntlm_auth): [2010/10/18 17:00:23,
> 0]
>> utils/ntlm_au
>> 977 Oct 18 17:00:23 rapsydprox2 (ntlm_auth): could not obtain
> winbind
>> domain name!
>> Following is my smb.conf
>>
>> workgroup = zzz
>> realm = zzz.COM
>> password server = *
>> #netbios name = zzz
>> server string = Samba file and print server
>> security = ADS
>> encrypt passwords = yes
>> log level = 2
>> log file = /var/log/samba/%m
>> max log size = 50
>> winbind separator = \.
>> #printcap name = cups
>> #printing = cups
>> ##new
>> idmap cache time = 1800
>> idmap negative cache time = 300
>> winbind cache time = 1800
>> ##end new
>> idmap uid = 15000-20000
>> idmap gid = 15000-20000
>> #winbind use default domain = yes
>> nt acl support = yes
>> map acl inherit = yes
>> winbind enum users = yes
>> winbind enum groups = yes
>> #client ntlmv2 auth = yes
>> template homedir = /home/%D/%U
>> template shell = /bin/bash
>> ldap passwd sync = yes
>>
>>
>>
>> Appreciate your help on this.
>>
>> Thanks,
>>
>> Tharanga
>>
>>
>> -----Original Message-----
>> From: Amos Jeffries
>>
>> On Mon, 18 Oct 2010 00:50:28 +0000, "Tharanga Abeyseela \(RGA\)" wrote:
>>> Hi Guys,
>>>
>>> I have been running squid with AD authentication and security group
>>> authent= ication for the last 6 months, and suddenly squid failed with
>> the
>>> following= message
>>>
>>> (squid): Too many queued ntlmauthenticator requests.
>>
>> Sign of a sudden burst of people logging in. Or that something caused a
>> slightly higher than normal delay in the auth checks. Which multiplies
> out
>> to fill up the Squid helpers.
>>
>>>
>>> Iam using
>>>
>>> samba-3.4.8-59.fc12.x86_64
>>> samba-winbind-clients-3.4.8-59.fc12.x86_64
>>> samba-common-3.4.8-59.fc12.x86_64
>>> samba-client-3.4.8-59.fc12.x86_64
>>> samba-winbind-devel-3.4.8-59.fc12.x86_64
>>> samba-winbind-3.4.8-59.fc12.x86_64
>>> Squid Cache: Version 3.0.STABLE24
>>> configure options: '--prefix=3D/usr/local/squid'
>>> '--enable-auth=3Dntlm,bas= ic'
>>> '--enable-external-acl-helpers=3Dwbinfo_group'
>>>
>>> and also iam getting the following error
>>>
>>> [2010/10/18 10:28:58, 3] winbindd/winbindd_cm.c:1597(connection_ok)
>>> connection_ok: Connection to for domain xxx is not connected
>>>
>>> (but if I use the net join command it says joined to domain)
>>>
>>> According to my squid report normally there are 700 people connecting
> to
>>> th= e proxy, but now its says winbind maximum connection exceeds 200,
>> but
>>> iam = confusing about that.
>>>
>>> Appreciate your help on this.
>>>
>>> Best regards,
>>> Tharanga
>>
>> Two basic design flaws in NTLM:
>> - winbind cannot accept more than ~256 simultaneous stateful logins (ie
>> NTLM)
>> - NTLM requires every new TCP connection to perform a full 2-request
>> handshake
>> (ie minimum login delays of request+reply+request+reply in HTTP)
>>
>> There are two things which each reduce the problem a little. When
> combined
>> they provide a great improvement in network speed and logins...
>>
>> - upgrade to 3.1.8.
>> - upgrade to Negotiate/Kerberos from NTLM
>>
>>
>> Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.8
   Beta testers wanted for 3.2.0.2