Re: [squid-users] Optimized Squids

On Tue, 22 Jun 2010 16:05:04 -0500, Seann Clark
<nombrandue_at_tsukinokage.net> wrote:
> All,
>
> I have been playing with/tweaking/breaking my squid for a few months

> now, and I am looking for suggestions from the list on improving
> performance. This is on a home system, which does not have a large user
> base. I am running a Dual Xeon 2.0 Ghz system with 2 gigs ram, 120 Gigs
> hard drive, in a Raid 5 configuration controlled by a 3ware RAID card.

Optimization Tip #1: get rid of the RAID-5.
  http://wiki.squid-cache.org/SquidFaq/RAID

> I was using the stock Fedora 8 RPM for this, which was single threaded,
> squid 2.6 Stable 22. I am also running this with diskd currently.

Optimization tip #2: run the fastest Squid release.
 Today 2.7 is current fastest official for production use.
 If you are willing to run semi-production code 2.HEAD daily snapshots
have more speed experiments and some few high-performance testers behind
it.

>
> I have recently recompiled squid to the latest stable for version
> 2.7 (Stable 9) with the async io flag passed to the configure command.
> After a little updating of my configurations, just enough to get it to
> work (I haven't changed any of the settings that are new to 2.7, so they

> are defaults right now), I have noticed a drastic improvement in speed
> and even when the system is at a high load (3-5 on the system load,

 So that covers #2. :)

Optimization tip #3: use fastest disk IO method available.
 Fedora being a linux that would be AUFS.

 Specifically in 2.7, AUFS tuned via cache_dir directive for large objects
(~1MB or more) alongside COSS directories for objects smaller than the AUFS
holds.
NP: these two caches types are happy to share one disk between them. An
exception to the on-disk-per-cache_dir rule of thumb.

> reported by top) it runs fairly well. I am looking at moving my cache
> directories off to a different disk, formatted with reiserFS, I am also
> planning on running it using diskd as the storage scheme. The drive I am

 see tip #3. AUFS makes use of multi-core threads, diskd process is
single-threaded even if it runs additional to the Squid main thread.

From my own experience, Squid has had no direct benefit from RieserFS over
extN. The OS and other apps sometimes do gain, so it's up to you.

You will need to tune the items which get journalled such as noatime etc
to increase disk IO speeds either way.

> using will be about 100 gigs of formatted space, and I plan to use all
> that space for the cache.

Make sure you have enough RAM for the indexes. 2GB is borderline for your
planned cache size.
4KB per object stored is what the index uses, with up to 2^31 objects per
cache_dir = up to 8GB index. Reality varies with your users habits.

NP: 2.7 in-memory handling of large objects is not great, so sacrificing
any large cache_mem RAM cache for COSS small-object cache is probably a
good trade-off in that release.

>
> I would like to know if this is a good plan, or should I change some

> things and how to change them, as well as any suggestions of
> configuration settings with the cache, and 2.7 options that may make a
> cache run even faster.
>

On top of those basic parameters, your ACL configuration strongly
influences speed. Read the wiki ACL pages on which are grouped as fast/slow
(NP: 'fast' does not strictly mean speed there) to get the ordering
optimal.
 Avoid any operations using regex like plague.
 Avoid helper operations as well since they often introduce large amounts
of lag as well.

Amos
Received on Wed Jun 23 2010 - 07:01:20 MDT