Re: [squid-users] Large scale high performance squid on FreeBSD - problems

On Fri, 5 Dec 2003, Andriy Korud wrote:

> Hi, I need to setup squid for approx 3000 clients on 30Mbit link.
> Estimated load is 10000reqs/min. Hardware is Xeon/2.8GHz, 1G RAM,
> Intel 1000/PRO Ethrenet and 2x36 Ultra320 SCSI disks dedicated for
> cache. OS is FreeBSD 4.9-STABLE, Squid 2.5STABLE4 installed from
> ports. Squid was configured for two storage directories using diskd.

You probably don't have enough SCSI disks to support that level of
requests, unless your cache_dir's are small enough to be heavily
cached in the 1GB of RAM you have, but even then that won't alleviate
the problem. Use at least several more SCSI disks, and spread the
cache over all of them in a JBOD (non-RAID) fashion using multiple
cache_dir directives. You could try just 5GB of cache per drive,
which doesn't seem like much, but if you make the cache too large then
Squid will consume large amounts of memory and have to manage too much
cache. 1GB of RAM can probably safely support no more than 50GB worth
of cache, but the processing requirements of that much cache might be
too high for your situation.

Also, check to make sure your SysV Shared Memory and Messaging
parameters are in line with what Squid requires to use diskd in your
situation. The formulas required for figuring out the diskd
requirements are in the Squid FAQ or other documentation, I can't
remember where. It appears, however, that FreeBSD now defaults to
much higher values than Squid requires in this area, so I should
probably remove the custom settings I set long ago for these since
they are actually now lower than the defaults. :-) If you want to
check your current values, just type 'sysctl kern.ipc' and you will
get a list of various IPC-related values, including all of the SysV
shared memory, messaging, and semaphore settings. You are only
concerned with about 8 of the almost 40 listed values, though.

You can also try this kernel option and see if it makes any
improvement:

options HZ=200 #Default is 100, which is 10ms/tick, 200 gives 5ms/tick

Setting HZ up to 1000 wouldn't be out of line for a very fast
single-CPU machine (such as a P4 or Xeon), which would give a tick
resolution of 1ms. I know this is especially important when dealing
with Dummynet for traffic shaping, and it also affects the process
scheduler. It might also affect other network-related operations
(poll()/select() ??) but I'm not sure. It would be nice to know if
higher HZ values have any effect on Squid under high-load situations.
Just keep in mind that the higher it is set, the more context switches
the kernel will perform per second, which can have an adverse effect
in some (mostly CPU-limited) situations.

-- 
 Chris Dillon - cdillon(at)wolves.k12.mo.us
 FreeBSD: The fastest, most open, and most stable OS on the planet
 - Available for IA32, IA64, PC98, Alpha, and UltraSPARC architectures
 - x86-64, PowerPC, ARM, MIPS, and S/390 under development
 - http://www.freebsd.org
Q: Because it reverses the logical flow of conversation.
A: Why is putting a reply at the top of the message frowned upon?