Re: [squid-users] Tuning Squid for large user base

I'm not sure about Linux but on BSD/OS and several of the other BSD OSes
you can improve performance by disabling hyperthreading when multiple Xeon
processors are present. The performance improvement seems to be around 15
percent. This eliminates the penalty for context switching in an SMP
environment.

Merton Campbell Crockett

On Fri, 5 Mar 2004, James MacLean wrote:

> Hi Folks,
>
> We have tried many suggestions that were found in mail-archives and on
> different sites, but are having a difficult time getting Squid to handle
> our workload. The obvious example of this to me is if we set
> "no_cache deny all", we can start going to sites and notice that they keep
> getting slower.
>
> The Squid server is a 2xP4 Xeon with hyperthreading 2.4Ghz with 1G of RAM.
> They have SCSI drives which we have tried various cache sizes on. There is
> no load on the server before squid begins, but it of course does drive up
> the CPU as it starts to churn.
>
> diskd seems to work the best, but once we start using it, the page load
> delays increase to a point that users are noticing. This setup is used in
> a transparent proxy via the typical Linux redirect.
>
> In this Educational setting we have approx. 12,000 active student/staff if
> I accept the "Largest file desc currently in use:" param, so we realize we
> need some power to deal with the actual disk cache effectiveness. There
> are actually more clients in total, but this seems to give a good guess of
> current activity.
>
> Our Internet feed is a 6Mbs max.
>
> Would like to ask if there are any tuning suggestions we can try to boost
> Squid in this environment?
>
> 2.5-STABLE5 cachemgr output follows for a short run :
>
> Connection information for squid:
> Number of clients accessing cache: 799
> Number of HTTP requests received: 261742
> Number of ICP messages received: 0
> Number of ICP messages sent: 0
> Number of queued ICP replies: 0
> Request failure ratio: 0.00
> Average HTTP requests per minute since start: 5769.8
> Average ICP messages per minute since start: 0.0
> Select loop called: 8599 times, 316.530 ms avg
> Cache information for squid:
> Request Hit Ratios: 5min: 17.9%, 60min: 13.7%
> Byte Hit Ratios: 5min: 3.8%, 60min: 1.9%
> Request Memory Hit Ratios: 5min: 25.1%, 60min: 30.8%
> Request Disk Hit Ratios: 5min: 32.0%, 60min: 24.9%
> Storage Swap size: 854948 KB
> Storage Mem size: 128036 KB
> Mean Object Size: 10.22 KB
> Requests given to unlinkd: 0
> Median Service Times (seconds) 5 min 60 min:
> HTTP Requests (All): 2.37608 1.71839
> Cache Misses: 2.79397 1.91442
> Cache Hits: 0.68577 0.61549
> Near Hits: 2.37608 1.91442
> Not-Modified Replies: 0.64968 0.49576
> DNS Lookups: 0.04048 0.03374
> ICP Queries: 0.00000 0.00000
> Resource usage for squid:
> UP Time: 2721.839 seconds
> CPU Time: 506.162 seconds
> CPU Usage: 18.60%
> CPU Usage, 5 minute avg: 78.16%
> CPU Usage, 60 minute avg: 18.46%
> Process Data Segment Size via sbrk(): 219948 KB
> Maximum Resident Size: 0 KB
> Page faults with physical i/o: 0
> Memory usage for squid via mallinfo():
> Total space in arena: 219948 KB
> Ordinary blocks: 219855 KB 375 blks
> Small blocks: 0 KB 0 blks
> Holding blocks: 6788 KB 3 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 92 KB
> Total in use: 226643 KB 100%
> Total free: 92 KB 0%
> Total size: 226736 KB
> Memory accounted for:
> Total accounted: 208789 KB
> memPoolAlloc calls: 38921604
> memPoolFree calls: 38222649
> File descriptor usage for squid:
> Maximum number of file descriptors: 32768
> Largest file desc currently in use: 11111
> Number of file desc currently in use: 3641
> Files queued for open: 1
> Available number of file descriptors: 29126
> Reserved number of file descriptors: 100
> Store Disk files open: 46
> Internal Data Structures:
> 88190 StoreEntries
> 17725 StoreEntries with MemObjects
> 17534 Hot Object Cache Items
> 83648 on-disk objects
>
> Lines from sample squid.conf:
>
> request_body_max_size 1000 MB
> httpd_accel_host virtual
> httpd_accel_port 80
> httpd_accel_with_proxy on
> httpd_accel_uses_host_header on
> icp_port 0
> cache_mem 128 MB
> cache_swap_low 90
> cache_swap_high 95
> cache_dir diskd /var/cache/squid/ 2000 64 64
> cache_access_log /var/log/squid/access.log
> cache_log /var/log/squid/cache.log
> cache_store_log /var/log/squid/store.log
> pid_filename /var/run/squid.pid
> debug_options ALL,1
> log_fqdn off
> redirect_children 30
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320
> quick_abort_min 16 KB
> quick_abort_max 16 KB
> quick_abort_pct 95
> forwarded_for off
> minimum_direct_hops 3
> store_avg_object_size 10 KB
> fqdncache_size 16536
> maximum_object_size_in_memory 8 KB
> request_header_max_size 1000 KB
> half_closed_clients off
>
>

-- 
BEGIN:				vcard
VERSION:			3.0
FN:				Merton Campbell Crockett
ORG:				General Dynamics Advanced Information Systems;
				Intelligence and Exploitation Systems
N:				Crockett;Merton;Campbell
EMAIL;TYPE=internet:		mcc@CATO.GD-AIS.COM
TEL;TYPE=work,voice,msg,pref:	+1(805)497-5045
TEL;TYPE=work,fax:		+1(805)497-5050
TEL;TYPE=cell,voice,msg:	+1(805)377-6762
END:				vcard