On 4/08/2013 7:13 p.m., John Joseph wrote:
> Thanks Augustus for the email
>
> my information is
>
> -------------------
>
> [root_at_proxy squid]# squidclient -h 127.0.0.1 mgr:storedir
> HTTP/1.0 200 OK
> Server: squid/3.1.10
> Mime-Version: 1.0
> Date: Sun, 04 Aug 2013 07:01:30 GMT
> Content-Type: text/plain
> Expires: Sun, 04 Aug 2013 07:01:30 GMT
> Last-Modified: Sun, 04 Aug 2013 07:01:30 GMT
> X-Cache: MISS from proxy
> X-Cache-Lookup: MISS from proxy:3128
> Via: 1.0 proxy (squid/3.1.10)
> Connection: close
>
> Store Directory Statistics:
> Store Entries : 13649421
> Maximum Swap Size : 583680000 KB
> Current Store Swap Size: 250112280 KB
> Current Capacity : 43% used, 57% free
>
> Store Directory #0 (aufs): /opt/var/spool/squid
> FS Block Size 4096 Bytes
> First level subdirectories: 32
> Second level subdirectories: 256
> Maximum Size: 583680000 KB
> Current Size: 250112280 KB
> Percent Used: 42.85%
> Filemap bits in use: 13649213 of 16777216 (81%)
> Filesystem Space in use: 264249784/854534468 KB (31%)
> Filesystem Inodes in use: 13657502/54263808 (25%)
> Flags: SELECTED
> Removal policy: lru
> LRU reference age: 44.69 days
You appear to have a good case there for upgrading to squid-3.2 or later
and adding a rock cache_dir.
As you can see 81% of the Filemap is full. That is the file number codes
Squid uses to internally reference stored objects. There is an absolute
limit of 2^24 (or "1677216" in the above report). That will require an
average object size of 35KB to fill your 557 GB storage area. Your
details earlier said the mean object size actually stored so far was 18KB.
If you add a 50GB rock store alongside that UFS directory you should be
able to double the cached object count.
> --------------
>
> and my squid.conf is as
>
> ----------------------------------------------
>
> always_direct allow all
> cache_log /opt/var/log/squid/cache.log
> cache_access_log /opt/var/log/squid/access.log
>
> cache_swap_low 90
> cache_swap_high 95
>
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32 ::1
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
>
> acl localnet src 172.16.5.0/24 # RFC1918 possible internal network
> acl localnet src 172.17.0.0/22 # RFC1918 possible internal network
> acl localnet src 192.168.20.0/24 # RFC1918 possible internal network
> acl localnet src fc00::/7 # RFC 4193 local private network range
> acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
> always_direct allow local-servers
You are using always_direct allow all above. This line is never even
being checked.
Also, always_direct has no meaning when there are no cache_peer lines to
be overridden (which is the purpose of always_direct). You can remove
both the always_direct lines to make things a bit faster.
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
>
> http_access allow manager localhost
> http_access deny manager
>
> http_access deny !Safe_ports
>
> http_access deny CONNECT !SSL_ports
>
>
> acl ipgroup src 172.16.5.1-172.16.5.255/32
> acl ipgroup src 172.17.0.10-172.17.3.254/32
> delay_pools 1
> delay_class 1 2
> delay_parameters 1 2560000/3860000 140000/180000
> delay_access 1 allow ipgroup
> delay_access 1 deny all
>
> http_access allow localnet
> http_access allow localhost
> http_access allow localnet
> http_access allow localhost
You have doubled these rules up.
> http_access deny all
>
> http_port 3128 transparent
It is a good idea to always have 3128 listing for regular proxy traffic
and redirecting the intercepted traffic to a separate port. The
interception port is a private detail only relevant to teh NAT
infrastructure doing the redirection and Squid. It can be firewalled to
prevent any access directly to the port.
>
> hierarchy_stoplist cgi-bin ?
>
> cache_dir aufs /opt/var/spool/squid 570000 32 256
>
> coredump_dir /opt/var/spool/squid
>
>
> maximum_object_size 4 GB
Can you try placing this above the cache_dir line please and see if it
makes any difference?
> refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private
>
> refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
> refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private
> refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
ignore-private and ignore-no-store are actually VERY bad ideas. No
matter that it looks okay for innocent things like images and archives.
Even those types are used in critical systems from time to time (think
security captchas using images, security certificates exchanged in
compressed archive formats, etc, etc).
Please remove them from the above lines. If you need them at all (eg to
fix a specific identifiable problem URL) it is best to target the regex
pattern to the specific domain or URLs.
In general the CMS systems and dynamic page frameworks use no-cache and
Expires to prevent unnecessary caching and force revalidation - 3.1 is
not fully capable of that but an upgrade to recent 3.2 or later releases
Squid can manage no-cache properly.
> refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
> refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
>
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 40% 40320
>
>
> visible_hostname proxy
This should be an FQDN if possible. The error page icons and similar
things will be reference at an http:// URL using the visible hostname as
domain and Squid forward-proxy port as port number. Your configuration
is probably sending traffic to "http://proxy:3128/", whatever that
resolves to in the client machine(s).
> icap_enable on
> icap_preview_enable on
> icap_preview_size 4096
> icap_persistent_connections on
> icap_send_client_ip on
> icap_send_client_username on
> icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod
> icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod
> adaptation_access qlproxy1 allow all
> adaptation_access qlproxy2 allow all
> ------------------------------------------------------------------------
> Guidance and advice requested
>
> Thanks for the reply
> Joseph John
>
Amos
Received on Sun Aug 04 2013 - 12:11:20 MDT
This archive was generated by hypermail 2.2.0 : Sun Aug 04 2013 - 12:00:14 MDT