Re: [squid-users] Performance Problem

Sven Eisenhauer wrote:

> Duane Wessels wrote:
>
>
>>>Storage: 160 GB external SCSI Raid Array (U2W-SCSI = 80 MB/s),
>>>256 MB RAM
>>>256 MB Swap
>>>Squid:
>>> cache_mem 32 MB
>>> maximum_object_size 1024 KB
>>> cache_dir ufs /var/squid/cache 156000 16 256
>>>
>>You have too much disk space and not enough RAM. Adjust
>>either so the ratio is about 32:1. You'll need
>>about 5GB of RAM for 156 GB of disk.
>>
>
> Thank you,
>
> but what should I do now?
> The RAID box was bought as Proxy Cache, but using only a few GB of 156
> GB does not make sense.

And having a box completely crippled by a gigantic Squid process makes
even less sense, doesn't it?

> But RAM for the Sparc is very expensive.
> Would it change something, if I use a number of smaller cache dirs?

No.

> Will asyncufs on the RAID destroy my existing cache? Where can I find
> information about Async I/O?

No, async ufs uses the same disk format as UFS and DiskD. I'm not sure
where there are complete docs on AUFS. The config file is a good place
to start, and my tuning guide talks about it a bit, though it talks
about Squid 2.2STABLE5+hno for now (I'll update it as soon as I get some
free time).

> By the way, why is Squid 2.4 always rebuilding my cache? This takes a
> few hours, which makes it very hard to quickly test something...

Because you have a tragically huge cache, and you've been shutting it
down before it finishes rebuilding (thus it has to rebuild again).
Normally, it will read in the swap.state files for each cache_dir. This
usually takes a few minutes on an unloaded cache with reasonably sized
cache_dirs. In your case, even reading in the swap is going to take a
pretty good while. On a heavily loaded cache, rebuilds can take
considerably longer (though I think in 2.4 this has been improved by
Robert and/or Kinkie?).

Decrease the size of your cache_dirs to something reasonable for 256MB
of RAM (like, say, 15-20GB total--though it's probably safe to go up to
30GB if performance doesn't matter much).

To put things into perspective for you, we ship our /lowest end/ server
with 256MB of RAM. This is a sub-$2,000 box with a single IDE disk,
configured to use ~12GB for cache object storage. Our highest end Squid
machine ships with 2GB of RAM and 4x18GB disks. You have to balance the
amount of RAM in the system to the amount of cache object storage you
want. We tend to be a little heavy on RAM, because performance matters
to our clients, but even so the balance we keep works very well and
provides excellent hit ratios.

While there are potential ways for Squid to not use so much memory (like
not keeping an in-core index), the current development path is not
taking us that way anytime soon. And the low cost of memory these days
prevents me from worrying about that fact too much.
                                   --
                      Joe Cooper <joe@swelltech.com>
                  Affordable Web Caching Proxy Appliances
                         http://www.swelltech.com
Received on Fri Sep 07 2001 - 02:31:09 MDT