Re: [squid-users] Re: DNS Queue Remains Filled Issue!

On 25/02/2013 7:47 p.m., arshan wrote:
> Hi,
>
> Thanks for response & sorry for late reply coz i have just recovered from
> fever....
>
>> Are they the same ones constantly? even after the timeout is reported to
> clients?
>
> No, the ids remains changing but all the clients keep on receiving timeout
> message. Here is first output of

Then the issue is not Squids DNS queuing at all. There _were_ bugs fixed
some years back where things would get 'stuck' in the queue and never
answered, but you are apparently not hitting those.

>
> #squidclient mgr:idns
> HTTP/1.0 200 OK
> Server: squid/3.1.20
> Mime-Version: 1.0
> Date: Mon, 25 Feb 2013 06:05:19 GMT
> Content-Type: text/plain
> Expires: Mon, 25 Feb 2013 06:05:19 GMT
> Last-Modified: Mon, 25 Feb 2013 06:05:19 GMT
> X-Cache: MISS from localhost
> X-Cache-Lookup: MISS from localhost:3128
> Via: 1.0 localhost (squid/3.1.20)
> Connection: close
>
> Internal DNS Statistics:
>
> The Queue:
> DELAY SINCE
> ID SIZE SENDS FIRST SEND LAST SEND
> ------ ---- ----- ---------- ---------
> 0xe1c6 42 5 40.115 10.043
> 0x4986 32 3 15.175 5.043
> 0xec04 46 5 31.429 1.043
> 0xdf69 31 7 107.340 37.043
> 0x0142 35 7 107.810 37.043
> 0x8f6d 33 7 97.736 27.043
> 0xa9c0 34 5 47.918 17.043
> 0x7f72 43 8 128.336 18.043
> 0xcbdd 31 8 113.573 3.043
> 0x171e 42 7 108.646 38.043
> 0x0322 40 1 3.502 3.502
> 0xb37c 41 7 104.886 34.043
> 0x92b0 33 5 44.396 14.043
> 0xad63 35 5 39.893 9.043
> 0x7fdc 36 4 24.085 4.043

This means that there are 15 DNS queries sent to server awaiting reply.
Most of them have been re-sent multiple times (up to 8 retries).

You can speed up the timeout cycle using
http://www.squid-cache.org/Doc/config/dns_retransmit_interval/ or
http://www.squid-cache.org/Doc/config/dns_timeout/

That won't solve your DNS problem, but will release any resources these
transactions are holding onto faster.

> Nameservers:
> IP ADDRESS # QUERIES # REPLIES
> ---------------------------------------------- --------- ---------
> 8.8.8.8 1415 1280
> 116.71.241.216 123 78

These are important. The numbers above should only differ by the count
of queued entries.
It means your DNS servers are actually not responding to about 1% of all
your DNS requests - just like the Squid error messages are saying.

> Rcode Matrix:
> RCODE ATTEMPT1 ATTEMPT2 ATTEMPT3
> 0 1357 0 0
> 1 0 0 0
> 2 0 0 0
> 3 1 0 0
> 4 0 0 0
> 5 0 0 0

Strangely the servers are almost lways responding with success when they
do respond. That is great - for the responses you get.

<snip>
>
>> Also try to debug why DNS is not responding immediately. Command line
> >from the Squid box:
>> dig -t AAAA example.com @<your DNS server>
>> dig -t A example.com @<your DNS server>
> Dig command shows that the dns are working fine, also the https queries are
> working well.

Yet a count of every single DNS request sent/received by Squid clearly
shows that 0.6-1% of them are not getting any response.

It is a pity we do not have the queue displaying what lookup each entry
is for. Then we might be able to find a pattern easily.

I am suspicious that you might be having Path-MTU discovery issues,
these can affect DNS UDP queries as well as TCP connections. Is ICMP
working fully along all networks between you and those resolvers?
  [ tracepath on Linux or mtupath on Windows are great tools for finding
that out quickly. Unless you happen to already known that ICMP blocking
being done.]

You could also locate the requests getting error messages and try a dig
for those domains using both UDP and TCP packets fro AAAA and A lookups.
If any of those fail to respond you have the answer.

Amos
Received on Tue Feb 26 2013 - 00:02:53 MST