Re: [squid-users] squid performance

On 23/10/10 03:01, Ananth wrote:
> Dear team,
>
> I run a Squid Cache: Version 3.1.8. i have a problem when my
> client_http.requests = is more than 200/sec. pages doesn't browse but
> when the request are less than 200 i dont find any problem. i don't
> see any errors in /etc/var/squid/cache.log. my file descriptors is
> 32768.
>
> Please find my configuration below and do suggest me if i m any where
> wrong in my configuration.

There is nothing visibly wrong with the below config. It's essentially
the default one which most are using happily.

I've pointed out a few bits which could be improved for overall speed,
but the gains are not ones which would suddenly cut in like that.

What does "squid -v" produce? and what OS is this on please?

>
> Thanks in advance.
>
> my h/w details is as fallows:
> CPU: 3.00 GHZ XEON processor
> RAM: 8 GB
> HDD: 148 GB * 2 SAS HDD
>
> my ulimint -n = 32768
>
> File descriptor usage for squid:
> Maximum number of file descriptors: 32768
> Largest file desc currently in use: 6064
> Number of file desc currently in use: 5656
> Files queued for open: 0
> Available number of file descriptors: 27112
> Reserved number of file descriptors: 100
> Store Disk files open: 119
>
> my squid.conf:
>
> ########### Start of squid.conf #created by ANANTH#############
> cache_effective_user squid
> cache_effective_group squid

effective-group is a piece of major voodoo with VERY limited real
use-cases. *general* recommendation is to trust the OS security settings
membership of "squid" user and remove that group option from the config.

>
> http_port 3128 transparent

With 3.1 this is now "intercept" to avoid confusion with tproxy
(transparent proxy).

>
> # httpd_accel_host virtual
> # httpd_accel_port 80
> # httpd_accel_with_proxy on
> # httpd_accel_uses_host_header on

Um, those should be removed.

 From your choice of "transparent" as a replacement I'm assuming you
want this as a transparent interception-proxy.
  If you want it as a reverse-proxy (what those old config lines did)
that is a whole separate config now.

>
> # cache_dir aufs /var/spool/squid 16384 32 512
> #--This has been inserted to check the cache--
> #cache_dir ufs /var/spool/squid 16384 16 256
> #cache_dir ufs /cache0/squid 16384 16 256
> #cache_dir ufs /squid0/squid 16384 16 256
> cache_dir aufs /squid1/squid 16384 32 512
> #cache_dir /tmp null
>
> cache_access_log /var/log/squid/access.log
> cache_log /var/log/squid/cache.log
> cache_store_log none
> logfile_rotate 7
> emulate_httpd_log on

Drop "emulate_httpd_log" and "cache_access_log".

Use this instead for the same output slightly faster:
   access_log /var/log/squid/access.log common

>
> cache_mem 3 GB
> maximum_object_size_in_memory 256 KB
> memory_replacement_policy lru
> cache_replacement_policy lru
> maximum_object_size 64 MB
>
> hierarchy_stoplist cgi-bin ?
> acl QUERY urlpath_regex cgi-bin \?
> no_cache deny QUERY

Drop the QUERY and cgi-bin stuff here. It will be forcing your Squid to
do slow network fetches for a lot of otherwise cacheable dynamic pages.
  There is a refresh_pattern below which fixes up the non-cacheable ones
behaviour.

>
> hosts_file /etc/hosts

Just a note:
   I've been seeing this in a lot of tutorials lately. This is not
needed unless you have a weird location for the hosts file (ie
/home/youraccount/hosts).
   There are ./configure options that should be used to integrate
correctly with the OS filesystem. This fixes a lot of file and folder
paths. Details in the squid wiki about each OS type.

>
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440

Add this right here:
   refresh_pattern -i (/cgi-bin/|\?) 0 0% 0

> refresh_pattern . 0 40% 4320
>
> ##Define your network below
>
> #acl mynetwork src 192.168.0.0/24
> acl mynetwork src 192.168.106.0/24 # cbinetwork private
> acl mynetwork src 192.168.107.0/24 # cbinetwork private
> acl mynetwork src 192.168.110.0/24 # cbinetwork private
> acl mynetwork src 192.168.120.0/24 # cbinetwork private
> acl mynetwork src 192.168.121.0/24 # cbinetwork private
> acl mynetwork src 192.168.130.0/24 # cbinetwork private
> acl mynetwork src 192.168.150.0/24 # cbinetwork private
> acl mynetwork src 192.168.151.0/24 # cbinetwork private
> acl mynetwork src 192.168.160.0/24 # cbinetwork private
> acl mynetwork src 10.100.101.0/24 # cbinetwork private
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32
> acl localhost src ::1/128
> acl to_localhost dst 127.0.0.0/8
> acl to_localhost dst ::1/128
> acl purge method PURGE
> acl CONNECT method CONNECT
>
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
>
> acl Safe_ports port 1025-65535 #unregistered ports
>
> acl SSL_ports port 443 563
>
> http_access allow manager localhost
> http_access deny manager
> http_access allow purge localhost
> http_access deny purge

Um do you actually need PURGE?
  If not remove it entirely from the config. Including the ACL
definition. Simply defining it makes Squid do more work tracking every
request.

> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
>
> http_access allow localhost
> http_access allow mynetwork
> # http_access deny all

"deny all" is the implicit final rule anyways. Leaving it configured
helps to make it explicitly clear where the denial happens.

> http_reply_access allow all
> icp_access allow mynetwork
>
> # icp_access deny all
>
> visible_hostname proxy.cbinet.bi
>
> coredump_dir /squid1/squid
>
> #
> ###############################################

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.8
   Beta testers wanted for 3.2.0.2