Re: me too (was Re: [SQU] Tune up Squid Cache Hit)

At 07:42 PM 2/26/01 +0800, M. Yu wrote:

>Hi all,
>
>Marc, thanks for the sample config. Something weird just came up. As I've
>mentioned I upgraded the hardware but still used the Squid that came with
>RH6.2 (2.3.STABLE1-5) and using the original squid.conf with the added
>sample you gave. Suddenly, instead of seeing a hit rate of 50% or so, my
>5min request hit ratio is now 1.1% and 60min is now 0.9%. My byte hit ratio
>is -5.2% for 5min and -2.5% for 60min. These are all under General
>Runtime's cache information for squid in cachemgr.cgi.

Somehow you lost your cache. It is hard to speculate from here why that
happened. If your cache is actually working you should see your cache
steadily growing over time (it might take a few weeks to stabilize). If you
do not see a steady growth of used disk space (df is fine) you have some
trouble. You might have a no-cache setting somewhere (or a implicit no
cache) Check your config thoroughly.

>What's also weird is that my /cache1 (formatted partition size 17Gb) which
>is configured to only be 60% used by Squid (10200 in cache_dir), is now 0%
>used (16Mb is used as reported by df -h) and doing an ls -lahR1 in /cache1
>reveals all cache subdirs are empty.
>
>I guess it's safe to assume my squid is NOT caching anything. Please help
>anyone. What do I need to look at? My bandwidth usage is hitting the roof
>and I can already almost see my ISP admin frowning upon our link bursting to
>over twice what we're paying for (256K burstable to 512K) at all hours of
>the day.
>
>I have included my squid.conf below without any changes (well except for the
>password). As you can see pretty much everything is default. The lines I
>added as indicated below was from a sample config Marc van Selm gave me to
>try and increase my hit rate to go close to 50%. My df -h output is also
>included below as well as cachemgr's output from General Runtime Info
>
>Thanks for any help guys!
>
>M. Yu
>
>
>--- df -h output ---
>
>[sa@cache1 19:46:52 ~]$ df -h
>Filesystem Size Used Avail Use% Mounted on
>/dev/hda5 2.9G 343M 2.4G 12% /
>/dev/hda1 7.6M 647k 6.5M 9% /boot
>/dev/sda1 17G 16M 16G 0% /cache1
>/dev/hda6 6.2G 710M 5.1G 12% /var
>
>
>--- output from cachemgr.cgi ---
>
>Squid Object Cache: Version 2.3.STABLE1
>
>Start Time: Sat, 24 Feb 2001 16:11:33 GMT
>Current Time: Mon, 26 Feb 2001 11:23:04 GMT
>
>Connection information for squid:
> Number of clients accessing cache: 79
> Number of HTTP requests received: 611966
> Number of ICP messages received: 19
> Number of ICP messages sent: 19
> Number of queued ICP replies: 0
> Request failure ratio: 0.00%
> HTTP requests per minute: 236.1
> ICP messages per minute: 0.0
> Select loop called: 9051723 times, 17.178 ms avg
>Cache information for squid:
> Request Hit Ratios: 5min: 1.1%, 60min: 0.9%
> Byte Hit Ratios: 5min: -5.2%, 60min: -2.5%
> Storage Swap size: 0 KB
> Storage Mem size: 11748 KB
> Storage Replacement Threshold: 0.000000
> Mean Object Size: 0.00 KB
> Requests given to unlinkd: 0
>Median Service Times (seconds) 5 min 60 min:
> HTTP Requests (All): 7.79421 7.38455
> Cache Misses: 7.79421 7.38455
> Cache Hits: 93.38594 5.63745
> Near Hits: 0.00000 3.86308
> Not-Modified Replies: 0.00000 0.00919
> DNS Lookups: 0.00278 0.00190
> ICP Queries: 0.00000 0.00000
>Resource usage for squid:
> UP Time: 155490.518 seconds
> CPU Time: 6469.270 seconds
> CPU Usage: 4.16%
> CPU Usage, 5 minute avg: 19.84%
> CPU Usage, 60 minute avg: 16.33%
> Maximum Resident Size: 0 KB
> Page faults with physical i/o: 845
>Memory usage for squid via mallinfo():
> Total space in arena: 30945 KB
> Ordinary blocks: 25020 KB 2711 blks
> Small blocks: 0 KB 0 blks
> Holding blocks: 176 KB 1 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 5925 KB
> Total in use: 25196 KB 81%
> Total free: 5925 KB 19%
>Memory accounted for:
> Total accounted: 19337 KB
>File descriptor usage for squid:
> Maximum number of file descriptors: 1024
> Largest file desc currently in use: 569
> Number of file desc currently in use: 512
> Files queued for open: 0
> Available number of file descriptors: 512
> Reserved number of file descriptors: 100
> Store Disk files open: 0
>Internal Data Structures:
> 2270 StoreEntries
> 2270 StoreEntries with MemObjects
> 2045 Hot Object Cache Items
> 0 Filemap bits set
> 0 on-disk objects
>
>
>--- start of SQUID.CONF ---
>
>
># NETWORK OPTIONS
># --------------------------------------------------------------------------
>---
>
>http_port 3128
>icp_port 3130
>
>
># OPTIONS WHICH AFFECT THE NEIGHBOR SELECTION ALGORITHM
># --------------------------------------------------------------------------
>---
>
>
>
># OPTIONS WHICH AFFECT THE CACHE SIZE
># --------------------------------------------------------------------------
>---
>
>cache_mem 12 MB
>cache_swap_low 94
>cache_swap_high 98
>maximum_object_size 3000000 KB
>ipcache_size 16384
>ipcache_low 90
>ipcache_high 95
>fqdncache_size 16384
>
>
># LOGFILE PATHNAMES AND CACHE DIRECTORIES
># --------------------------------------------------------------------------
>---
>
>cache_dir ufs /cache1 10200 16 256
>cache_access_log /var/log/squid/access.log
>cache_log /var/log/squid/cache.log
>cache_store_log none
>emulate_httpd_log off
>mime_table /etc/squid/mime.conf
>log_mime_hdrs on
>useragent_log /var/log/squid/useragent.log
>pid_filename /var/run/squid.pid
>debug_options ALL,1
>log_fqdn on
>client_netmask 255.255.255.255
>
>
># OPTIONS FOR EXTERNAL SUPPORT PROGRAMS
># --------------------------------------------------------------------------
>---
>
>ftp_user Squid@
>ftp_list_width 64
>cache_dns_program /usr/lib/squid/dnsserver
>dns_children 10
>dns_defnames off
>unlinkd_program /usr/lib/squid/unlinkd
>
>
># OPTIONS FOR TUNING THE CACHE
># --------------------------------------------------------------------------
>---
>
>request_header_max_size 10 KB
>request_body_max_size 7 MB
>reply_body_max_size 0
>replacement_policy LFUDA
>negative_ttl 5 minutes
>positive_dns_ttl 6 hours
>negative_dns_ttl 5 minutes
>range_offset_limit 0 KB
>
>
># TIMEOUTS
># --------------------------------------------------------------------------
>---
>
>connect_timeout 120 seconds
>peer_connect_timeout 30 seconds
>siteselect_timeout 4 seconds
>read_timeout 15 minutes
>request_timeout 30 seconds
>client_lifetime 1 day
>half_closed_clients off
>pconn_timeout 120 seconds
>ident_timeout 10 seconds
>shutdown_lifetime 30 seconds
>
>
># ACCESS CONTROLS
># --------------------------------------------------------------------------
>---
>
>acl cable_subscribers src 64.110.88.0/255.255.255.128
>acl servers src 64.110.82.80/255.255.255.240
>acl all src 0.0.0.0/0.0.0.0
>acl manager proto cache_object
>acl localhost src 127.0.0.1/255.255.255.255
>acl SSL_ports port 443 563
>acl Safe_ports port 80 21 443 563 70 210 1025-65535
>acl Safe_ports port 280 # http-mgmt
>acl Safe_ports port 488 # gss-http
>acl Safe_ports port 591 # filemaker
>acl Safe_ports port 777 # multiling http
>acl CONNECT method CONNECT
>http_access allow cable_subscribers
>http_access allow manager servers
>http_access allow manager localhost
>http_access deny manager
>http_access deny !Safe_ports
>http_access deny CONNECT
>http_access allow localhost
>http_access deny all
>icp_access deny all
>miss_access allow all
>ident_lookup_access deny all
>
>
># ADMINISTRATIVE PARAMETERS
># --------------------------------------------------------------------------
>---
>
>cache_mgr cachemgr@websprinter.net
>cache_effective_user squid
>cache_effective_group squid
>visible_hostname cache.websprinter.net
>unique_hostname cache1.websprinter.net
>
>
># MISCELLANEOUS
># --------------------------------------------------------------------------
>---
>
>logfile_rotate 10
>append_domain .websprinter.net
>tcp_recv_bufsize 0 bytes
>memory_pools on
>forwarded_for off
>log_icp_queries off
>icp_hit_stale off
>minimum_direct_hops 1
>cachemgr_passwd whatever all
>store_avg_object_size 13 KB
>store_objects_per_bucket 50
>client_db on
>netdb_low 900
>netdb_high 1000
>query_icmp off
>test_reachability off
>buffered_logs off
>acl local-servers dstdomain websprinter.net
>always_direct allow local-servers
>minimum_retry_timeout 5 seconds
>maximum_single_addr_tries 3
>snmp_port 0
>uri_whitespace strip
>ignore_unknown_nameservers on
>
>
># Added by MYu on 2000-02-21
>
># Don't cache these.
>acl QUERY url_regex cgi-bin \?
>no_cache deny QUERY
>
># Handle dynamic html (don't cache and always go direct)
>acl DHTML urlpath_regex \.(jhtml|phtml|shtml) \.asp \? cgi-bin \.cgi \.pl
>always_direct allow DHTML
>no_cache deny DHTML
>
>
># Special cases to improve mega-sites
># News sites
>refresh_pattern ^http://.*\.cnn\.com 360 50% 4320 override-lastmod
>override-expire ignore-reload
>refresh_pattern ^http://news\.bbc\.co\.uk 360 50% 4320 override-lastmod
>override-expire ignore-reload
># Inefficient sites
>refresh_pattern microsoft 1080 150% 10080 override-expire override-lastmod
>override-expire
>refresh_pattern msn\.com 4320 150% 10080 override-expire override-lastmod
>override-expire
># Adds from doublclick
>refresh_pattern ^http://.*\.doubleclick\.net 10080 300% 40320
>override-expire override-lastmod override-expire ignore-reload
># Pointcast
>refresh_pattern ^http://.*FIDO 360 1000% 480
>refresh_pattern \.phtml$ 240 200% 10080
>#
># Audio:
># these min 7 days, 150% of age, max 28 days (not likely to change)
>refresh_pattern \.avi$ 10080 150% 40320
>refresh_pattern \.mov$ 10080 150% 40320
>refresh_pattern \.wav$ 10080 150% 40320
>refresh_pattern \.mp3$ 10080 150% 40320
>refresh_pattern \.qtm$ 10080 150% 40320
>refresh_pattern \.mid$ 10080 150% 40320
>refresh_pattern \.viv$ 10080 150% 40320
>refresh_pattern \.mpg$ 10080 150% 40320
>refresh_pattern \.jpg$ 10080 150% 40320
>refresh_pattern \.rar$ 10080 150% 40320
>refresh_pattern \.ram$ 10080 150% 40320
>refresh_pattern \.r[0-9][0-0]$ 10080 150% 40320
>#
># gif are most likely in html lets do them a bit less (more likely)
># asis are most likely in html lets do them a bit less (more likely)
># min 1 day, 50% of age and max 14 days
>refresh_pattern ^http://.*\.gif$ 1440 50% 20160
>refresh_pattern ^http://.*\.asis$ 1440 50% 20160
># other gifs a bit more. (ftp downloaded images?)
>refresh_pattern \.gif$ 10080 300% 40320
>#
># Txt
>refresh_pattern \.txt$ 1440 100% 20160 override-lastmod
>#
># Downloads: these PC-things
># min 2 days, 200% of age, max 28 days (not so likely to change)
>refresh_pattern \.zip$ 2880 200% 40320
>refresh_pattern \.arj$ 2880 200% 40320
>refresh_pattern \.exe$ 2880 200% 40320
>#
># Unix distributions: not likely at all to change:
># min 7 days, 200% of age, max 28 days
>refresh_pattern \.tgz$ 10080 200% 40320
>refresh_pattern \.gz$ 10080 200% 40320
>refresh_pattern \.tgz$ 10080 200% 40320
>refresh_pattern \.tar$ 10080 200% 40320
>refresh_pattern \.Z$ 10080 200% 40320
>#
># Other stuff
># PDF min 6 hours, 100% of last modified max 28 days and enforce min
>(override-lastmod)
>refresh_pattern \.pdf$ 720 100% 40320 override-lastmod
>#
># keep ftp/gopher (mainly directories) fresh for 1 day, 50%, max 10 days
>refresh_pattern ^ftp:// 1440 50% 14400 override-lastmod
>refresh_pattern ^gopher:// 1440 50% 14400 override-lastmod
>#
># all the rest 6 hours, 50% of age, max 5 days
>refresh_pattern . 360 50% 7200 override-lastmod
>
>#
>quick_abort_min 100 KB
>quick_abort_max 750 KB
>quick_abort_pct 60
>
>#
>reload_into_ims on
>
>
>--
>To unsubscribe, see http://www.squid-cache.org/mailing-lists.html

--------------------------------------------------------------------
Marc van Selm
NATO C3 Agency
Communication Systems Division, A-Branch
Tel: +31 70 3142454
E-mail: marc.van.selm@nc3a.nato.int (PGP capable)
--------------------------------------------------------------------
Private: selm@cistron.nl, selm@het.net, http://www.cistron.nl/~selm

--
To unsubscribe, see http://www.squid-cache.org/mailing-lists.html