Re: HELP!!! Large Site with Squid problem..!!!

What tuning have you applied to your Linux server?

a) Maximum number of open filedescriptors
b) SYN queue length?
c) IP port range?
d) Maximum number of active inodes/files?
e) Have you enabled async-io?

As you say nothing gets logged my suspision is (a) or (b) is becoming
your bottleneck.

a) When Squid approaches the amount of configured filedescriptors it
will stop accepting new connections. See the cachemgr pages for details
on your current usage.
See http://squid.sourceforge.net/hno/ for details on how to increase the
limit.

b) When you have many high latency clients (i.e. dialup or WAN) quite
many will be in the TCP SYN handshake phase blocking positions in the
SYN queue. If more than 128 clients are in the SYN handshake then no
further clients will be able to initiate sessions to the proxy service
until some of the pending ones have completed the handshake and has been
accepted by Squid.
This limit is tuned in /proc/sys/net/ipv4/tcp_max_syb_backlog

c) When the outgoing request rate is high. TCP will soon run out of
available ports for the local end of the connections. The default is
only ~4000 ports, and each outgoing connection listed by netstat -an
uses up one port. (including those in TIME_WAIT state).
Quick measurement:
  netstat -an | awk '/^tcp/ {print $4}'| cut -d: -f2 | sort -u | wc -l

Squid should complain in cache.log when the limit is reached.

This limit is tuned in /proc/sys/net/ipv4/ip_local_port_range

d) When these limits are reached no more files or sockets can be opened.
Both Linux and Squid complains quite heavily when it happens.
These limits are tuned in /proc/sys/fs/max-files and max-inodes

e) Without async-io Squid will not be able to handle any higher request
loads as it's speed will basically be limited to the seek speed of a
single drive.
async-io is enabled in Squid by the --enable-async-io configure option.
My recommended Squid version for running async-io is
Squid-2.2.STABLE5-hno (see
http://squid.sourceforge.net/hno/patch-2.2.html)

And I hope you are not running RAID5 on that raid controller? Doin so
degradedes the performance of the raid set to that equal of a single
drive. Squids I/O pattern is random I/O of small blocks of data, i.e.
the worst possible for RAID5.

--
Henrik Nordstrom
Squid hacker
Daniel Evans wrote:
> We run a large ISP in New Zealand and are having TREMENDOUS problems
> with our Cache server.. (Squid Version 2.2 under Redhat Linux 6.1)
>
> We find that when we pass a fiar amount of traffic through them Squid
> keeps running but slowly stops serving pages to anyone.. Eventually it
> stops running and leaves no comments in the error logs as to why. Both
> of the machines that we have do this...