[squid-users] Tuning Squid for large user base

Hi Folks,

We have tried many suggestions that were found in mail-archives and on
different sites, but are having a difficult time getting Squid to handle
our workload. The obvious example of this to me is if we set
"no_cache deny all", we can start going to sites and notice that they keep
getting slower.

The Squid server is a 2xP4 Xeon with hyperthreading 2.4Ghz with 1G of RAM.
They have SCSI drives which we have tried various cache sizes on. There is
no load on the server before squid begins, but it of course does drive up
the CPU as it starts to churn.

diskd seems to work the best, but once we start using it, the page load
delays increase to a point that users are noticing. This setup is used in
a transparent proxy via the typical Linux redirect.

In this Educational setting we have approx. 12,000 active student/staff if
I accept the "Largest file desc currently in use:" param, so we realize we
need some power to deal with the actual disk cache effectiveness. There
are actually more clients in total, but this seems to give a good guess of
current activity.

Our Internet feed is a 6Mbs max.

Would like to ask if there are any tuning suggestions we can try to boost
Squid in this environment?

2.5-STABLE5 cachemgr output follows for a short run :

Connection information for squid:
        Number of clients accessing cache: 799
        Number of HTTP requests received: 261742
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 5769.8
        Average ICP messages per minute since start: 0.0
        Select loop called: 8599 times, 316.530 ms avg
Cache information for squid:
        Request Hit Ratios: 5min: 17.9%, 60min: 13.7%
        Byte Hit Ratios: 5min: 3.8%, 60min: 1.9%
        Request Memory Hit Ratios: 5min: 25.1%, 60min: 30.8%
        Request Disk Hit Ratios: 5min: 32.0%, 60min: 24.9%
        Storage Swap size: 854948 KB
        Storage Mem size: 128036 KB
        Mean Object Size: 10.22 KB
        Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 2.37608 1.71839
        Cache Misses: 2.79397 1.91442
        Cache Hits: 0.68577 0.61549
        Near Hits: 2.37608 1.91442
        Not-Modified Replies: 0.64968 0.49576
        DNS Lookups: 0.04048 0.03374
        ICP Queries: 0.00000 0.00000
Resource usage for squid:
        UP Time: 2721.839 seconds
        CPU Time: 506.162 seconds
        CPU Usage: 18.60%
        CPU Usage, 5 minute avg: 78.16%
        CPU Usage, 60 minute avg: 18.46%
        Process Data Segment Size via sbrk(): 219948 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
        Total space in arena: 219948 KB
        Ordinary blocks: 219855 KB 375 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 6788 KB 3 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 92 KB
        Total in use: 226643 KB 100%
        Total free: 92 KB 0%
        Total size: 226736 KB
Memory accounted for:
        Total accounted: 208789 KB
        memPoolAlloc calls: 38921604
        memPoolFree calls: 38222649
File descriptor usage for squid:
        Maximum number of file descriptors: 32768
        Largest file desc currently in use: 11111
        Number of file desc currently in use: 3641
        Files queued for open: 1
        Available number of file descriptors: 29126
        Reserved number of file descriptors: 100
        Store Disk files open: 46
Internal Data Structures:
         88190 StoreEntries
         17725 StoreEntries with MemObjects
         17534 Hot Object Cache Items
         83648 on-disk objects

Lines from sample squid.conf:

request_body_max_size 1000 MB
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
icp_port 0
cache_mem 128 MB
cache_swap_low 90
cache_swap_high 95
cache_dir diskd /var/cache/squid/ 2000 64 64
cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
pid_filename /var/run/squid.pid
debug_options ALL,1
log_fqdn off
redirect_children 30
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
quick_abort_min 16 KB
quick_abort_max 16 KB
quick_abort_pct 95
forwarded_for off
minimum_direct_hops 3
store_avg_object_size 10 KB
fqdncache_size 16536
maximum_object_size_in_memory 8 KB
request_header_max_size 1000 KB
half_closed_clients off
Received on Fri Mar 05 2004 - 11:47:55 MST