Re: [squid-users] Re: Re: kerberos auth failing behind a load balancer

From: Sean Boran <sean_at_boran.com>
Date: Thu, 14 Mar 2013 17:10:23 +0100

Markus,

The klist outputs are further below, but I have the feeling that is
not the problem, that the solution needs to be different (after
reading the following articles).

See for example:
"the thing to watch out for is that AD will fail to return a ticket if
the SPN requested is found on more than one account (because it
doesn't know which account to use). So be careful that you do not
accidentally create multiple service accounts with the same SPN."
http://newsgroups.derkeiler.com/Archive/Comp/comp.protocols.kerberos/2009-09/msg00029.html

and for Windows services, SPNs are linked to a user:
"To use Kerberos authentication with a load-balanced array of Client
Access servers ..All computers within the Client Access server array
must share the same service account... You can create a computer
account or a user account for the alternate service account"
A user account is created, then the load balanced SPNs are associated
with that account
Setspn -S exchangeMDB/outlook.corp.contoso.com
contoso\newSharedServiceAccountName
http://technet.microsoft.com/en-us/library/ff808312.aspx
http://blogs.technet.com/b/exchange/archive/2011/04/15/recommendation-enabling-kerberos-authentication-for-mapi-clients.aspx

So now I've ordered a user account in the AD (this takes a while
internally here) and will try and link the proxy.example SPN to that,
and copy the keytab to each proxy.
Something like this:
Kerberos Authentication on Load Balanced Web Servers
http://trialanderrorlinux.wordpress.com/2012/06/13/29/
http://trialanderrorlinux.wordpress.com/2012/06/11/setting-up-kerberos-authentication-for-app-and-web/

--snip--
proxy2:~# klist -ekt /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp Principal
---- ---------------- ---------------------------------------------------------
   5 20/02/2013 15:14 HTTP/proxy2.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 20/02/2013 15:14 HTTP/proxy2.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 20/02/2013 15:14 HTTP/proxy2.example.com_at_EXAMPLE.NET (arcfour-hmac)
   5 20/02/2013 15:14 HTTP/proxy2_at_EXAMPLE.NET (des-cbc-crc)
   5 20/02/2013 15:14 HTTP/proxy2_at_EXAMPLE.NET (des-cbc-md5)
   5 20/02/2013 15:14 HTTP/proxy2_at_EXAMPLE.NET (arcfour-hmac)
   5 20/02/2013 15:14 HTTP/proxy.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 20/02/2013 15:14 HTTP/proxy.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 20/02/2013 15:14 HTTP/proxy.example.com_at_EXAMPLE.NET (arcfour-hmac)
   5 20/02/2013 15:14 HTTP/proxy.example.com/proxy2_at_EXAMPLE.NET (des-cbc-crc)
   5 20/02/2013 15:14 HTTP/proxy.example.com/proxy2_at_EXAMPLE.NET (des-cbc-md5)
   5 20/02/2013 15:14
HTTP/proxy.example.com/proxy2.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 20/02/2013 15:14
HTTP/proxy.example.com/proxy2.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 20/02/2013 15:14 HTTP/proxy.example.com/proxy2_at_EXAMPLE.NET (arcfour-hmac)
   5 20/02/2013 15:14
HTTP/proxy.example.com/proxy2.example.com_at_EXAMPLE.NET (arcfour-hmac)

proxy3:~# klist -ekt /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp Principal
---- ---------------- ---------------------------------------------------------
   5 11/03/2013 16:11 host/proxy3.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:11 host/proxy3.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:11 host/proxy3.example.com_at_EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:11 host/proxy3_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:11 host/proxy3_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:11 host/proxy3_at_EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:11 proxy3$@EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:11 proxy3$@EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:11 proxy3$@EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:12
HTTP/proxy3.example.com/proxy3.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:12
HTTP/proxy3.example.com/proxy3.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:12
HTTP/proxy3.example.com/proxy3.example.com_at_EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:12 HTTP/proxy3.example.com/proxy3_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:12 HTTP/proxy3.example.com/proxy3_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:12 HTTP/proxy3.example.com/proxy3_at_EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:13 HTTP/proxy.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:13 HTTP/proxy.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:13 HTTP/proxy.example.com_at_EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:32
HTTP/proxy.example.com/proxy3.example.com_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:32
HTTP/proxy.example.com/proxy3.example.com_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:32
HTTP/proxy.example.com/proxy3.example.com_at_EXAMPLE.NET (arcfour-hmac)
   5 11/03/2013 16:32 HTTP/proxy.example.com/proxy3_at_EXAMPLE.NET (des-cbc-crc)
   5 11/03/2013 16:32 HTTP/proxy.example.com/proxy3_at_EXAMPLE.NET (des-cbc-md5)
   5 11/03/2013 16:32 HTTP/proxy.example.com/proxy3_at_EXAMPLE.NET (arcfour-hmac)
--/snip--

Gruss,

Sean

On 13 March 2013 20:49, Markus Moeller <huaraz_at_moeller.plus.com> wrote:
> Hi Sean,
>
> Can you do a klist -ekt <squid.keytab> on both squid servers and send me
> the output ? I assume you are missing entries.
>
> Markus
>
> "Sean Boran" <sean_at_boran.com> wrote in message
> news:CAOnghjtWpc0fPBVVB=yF3BeGLgfRrf1jQOXLzVBFHuhbVYLo0Q_at_mail.gmail.com...
>
> (sorry for the slow answer, an over-eager spam filter swallowed this msg).
>
> In wireshark, the server name sent in the ticket is correct
> (proxy.example.com) , encryption is rc4-hmac and knvo=5.
> This is the same kvno as seen in "klist -ekt /etc/krb5.keytab" (with
> des-cbc-crc, des-cbc-md5, arcfour-hmac).
>
> Now there are two squids behind the balancer; one of them will behave
> correctly and accept kerberos authentication to the balanced proxy
> name. (I had not realised the second one worked before). Comparing the
> quid and kerb config does not explain the difference.
>
> However on a windows client, querying SPN for the balanced name only
> lists the squid proxy that works(proxy2) and no mention of proxy3.
>
> C:\temp>cscript spn_query.vbs http/proxy.example.com example.net
> CN=proxy2,OU=Ubuntu,OU=Server,..
> O,DC=example,DC=net
> Class: computer
> Computer DNS: proxy2.example.com
> -- http/proxy.example.com
> -- HTTP/proxy.example.com/proxy2
> -- HTTP/proxy.example.com/proxy2.example.com
> -- HTTP/proxy2
> -- HTTP/proxy2.example.com
> -- HOST/proxy2.example.com
> -- HOST/PROXY2
>
> Next, tried to use the windows tool setspn to add an spn for proxy3:
> setspn -S http/proxy.example.com proxy3
> but it says "Duplicate SPN found, aborting operation!"
> which makes me think I'm misunderstanding. Its is not possible to
> assign the same SPN to real names of both the squids behind the
> balancer?
>
> Thanks,
>
> Sean
>
>
> On 1 March 2013 21:06, Markus Moeller <huaraz_at_moeller.plus.com> wrote:
>>
>> That should work. What do you see in Wireshark when you look at the
>> traffic
>> to the proxy ? If you exand the Negotiate header you should see what is
>> the
>> principal name and kvno. Both must match what is in your keytab ( check
>> with
>> klist -ekt /etc/keytab)
>>
>> Markus
>>
>>
>> "Sean Boran" <sean_at_boran.com> wrote in message
>> news:CAOnghjuyE0OYooMKquWL5fRMNyozfrVuEksLbNXYAO0kEL_QAA_at_mail.gmail.com...
>>
>> Hi,
>>
>> I’ve received (kemp) load balancers to put in front of squids to
>> provide failover.
>> The failover / balancing works fine until I enable Kerberos auth on the
>> squid.
>>
>> Test setup:
>> Browser ==> Kemp balancer ==> Squid ==>
>> Internet
>> proxy.example.com proxy3.example.com
>>
>> The client in Windows7 in an Active Directory domain.
>> If the browser proxy is set to proxy3.example.com (bypassing the LB),
>> Kerberos auth works just fine, but via the kemp (proxy.example.com)
>> the browser prompts for a username/password which is not accepted
>> anyway
>>
>> Googling on Squid+LBs, the key is apparently to add a principal for the
>> LB,
>> e.g.
>> net ads keytab add HTTP/proxy.example.com
>>
>> In the logs (below), one can see the client sending back a Krb ticket
>> to squid, but it rejects it:
>> "negotiate_wrapper: Return 'BH gss_accept_sec_context() failed:
>> Unspecified GSS failure. "
>> When I searched on that. one user suggested changing the encryption in
>> /etc/krb5.conf . In /etc/krb5.conf I tried with the recommended
>> squid settings (see below), and also with none at all. The results
>> were the same. Anyway, if encryption was the issue, it would not work,
>> via LB or directly.
>>
>>
>> Analysis:
>> -------------
>> When the client sent a request, squid replies with:
>>
>> HTTP/1.1 407 Proxy Authentication Required
>> Server: squid
>> X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
>> X-Cache: MISS from proxy3.example.com
>> Via: 1.1 proxy3.example.com (squid)
>>
>> ok so far. the client answer with a kerberos ticket:
>>
>> Proxy-Authorization: Negotiate YIIWpgYGKwYBXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>>
>> UserRequest.cc(338) authenticate: header Negotiate
>> YIIWpgYGKwYBXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>> UserRequest.cc(360) authenticate: No connection authentication type
>> Config.cc(52) CreateAuthUser: header = 'Negotiate YIIWpgYGKwYBBQUCXXXX
>> auth_negotiate.cc(303) decode: decode Negotiate authentication
>> UserRequest.cc(93) valid: Validated. Auth::UserRequest '0x20d68d0'.
>> UserRequest.cc(51) authenticated: user not fully authenticated.
>> UserRequest.cc(198) authenticate: auth state negotiate none. Received
>> blob: 'Negotiate
>> YIIWpgYGKwYBBQUCoIIWmjCCFpagMDAuBgkqhkiC9xIBAXXXXXXXXXX
>> ..
>> UserRequest.cc(101) module_start: credentials state is '2'
>> helper.cc(1407) helperStatefulDispatch: helperStatefulDispatch:
>> Request sent to negotiateauthenticator #1, 7740 bytes
>> negotiate_wrapper: Got 'YR YIIWpgYGKwYBBQXXXXXXXXXXXXXXX
>> negotiate_wrapper: received Kerberos token
>> negotiate_wrapper: Return 'BH gss_accept_sec_context() failed:
>> Unspecified GSS failure. Minor code may provide more information.
>>
>>
>> Logs for a (successful) auth without LB:
>> .. as above ....
>> negotiate_wrapper: received Kerberos token
>> negotiate_wrapper: Return 'AF oYGXXXXXXXXXXXXXXXXXXXXXXA==
>> USER_at_EXAMPLE.NET
>>
>>
>> ----- configuration ---
>> Ubuntu 12.04 + std kerberod. Squid 3.2 bzr head from lat Jan.
>> - squid.conf:
>> - debug_options ALL,2 29,9 (to catch auth)
>> auth_param negotiate program
>> /usr/local/squid/libexec/negotiate_wrapper_auth -d --kerberos
>> /usr/local/squid/libexec/negotiate_kerberos_auth -s GSS_C_NO_NAME
>> --ntlm /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
>> auth_param negotiate children 20 startup=20 idle=20 auth_param
>> negotiate keep_alive on
>>
>> - The LB is configured as a Generic Proxy (does not try to interpret
>> the HTTP stream), with with Layer 7 transparency
>> (it forwards traffic to the squid, the squid see the real client IP,
>> and squid traffic is routed back though the LB)
>> I've tried playing with the LB Layer 7 settings, to no avail.
>>
>> Samba:
>> net ads join -U USER
>> net ads testjoin
>> Join is OK
>>
>> net ads keytab add HTTP -U USER
>> net ads keytab add HTTP/proxy.example.com -U USER
>> chgrp proxy /etc/krb5.keytab
>> chmod 640 /etc/krb5.keytab
>> strings /etc/krb5.keytab # check contents
>> net ads keytab list
>>
>> /etc/krb5.conf
>> [libdefaults]
>> default_realm = EXAMPLE.NET
>> kdc_timesync = 1
>> ccache_type = 4
>> forwardable = true
>> proxiable = true
>> fcc-mit-ticketflags = true
>> default_keytab_name = FILE:/etc/krb5.keytab
>> dns_lookup_realm = no
>> ticket_lifetime = 24h
>>
>> [realms]
>> EXAMPLE.net = {
>> kdc = ldap.EXAMPLE.net
>> master_kdc = ldap.EXAMPLE.net
>> admin_server = ldap.EXAMPLE.net
>> default_domain = EXAMPLE.net
>> }
>> [domain_realm]
>> .corproot.net = EXAMPLE.NET
>> corproot.net = EXAMPLE.NET
>>
>>
>> Any suggestions on where I could dig further?
>>
>> Thanks in advance,
>>
>> Sean Boran
>>
>>
>
>
Received on Thu Mar 14 2013 - 16:10:30 MDT

This archive was generated by hypermail 2.2.0 : Fri Mar 15 2013 - 12:00:05 MDT