Re: [squid-users] Metrics to calculate 'best' values for cache_mem and cache_dir?

On 03/27/2012 10:42 AM, Peter Gaughran wrote:
> With the recent (enlightening) posts from Christian, Marcus et al, I was wondering what the metrics actually are for calculating cache_mem and cache_dir? I've read (for Linux at least) aufs is
> superior to diskd, so I've made the appropriate decision, but any time any reference is made to size of caches or directory depth, figures are mentioned that seem to be obeying formulae, but I've yet
> to find the actual formulae? Are they any other critical parameters I should be wary of, resource wise?
>
> I've had a look over the Squid wiki, and various other sources, but can't seem to find a definitive answer? Apologies if I'm being daft and missing something that's well known :)
>
> Our current setup is a 32 gig, single CPU (E5620) with 4 15k 300 gig disks available for disk cache (o/s is elsewhere).

I think there is no one-formula-fits-all since sites have different
user groups which visit different sets of websites which have
different caching patterns.

Having said that, there are rules of thumb.

There are various trade-offs:
memory-cache vs disk-cache vs no-cache (retrieve again)
speed vs investment

no-cache can be faster than disk-cache but needs more bandwidth
memory-cache is faster than disk-cache and no-cache

 From the above is seems logical to have a large memory-cache.
Prices and specs of bandwidth vary. Based on your bandwidth
you can decide where to invest: more bandwidth or very fast disks.
If bandwidth is cheap, a server with only a memory cache and
no disk cache can be a proper configuration.

Squid is CPU bound and up to version 3.1.x is bound by the speed
of a single CPU core. The not yet released 3.2 uses multiple
cores. So currently, the faster the CPU, the better and the more
cache a CPU has, the better.

In your particular case with 4 high speed disks and 32 GB memory
you could choose for
4 aufs cache directories: 1 per disk
total disk cache of 300 GB (4,2 GB in-memory index)
memory-cache of 22 GB
This leaves 6 GB for all other resources:
- other memory usage by Squid
- other processes
- OS including file system buffers and network buffers

You need a very large swap for squid to be able to fork.
At least 32 GB swap but I think that 64 GB is better.

This is only a suggested setup. Always look carefully at
the system behavior, especially swap.

Marcus
Received on Tue Mar 27 2012 - 17:31:11 MDT