Re: [squid-users] squid slow

On Mon, 24 Aug 2009 10:24:41 +0600, Muhammad Sharfuddin
<m.sharfuddin_at_nds.com.pk> wrote:
> Squid Version 2.7.STABLE5
> SUSE Enterprise 11 i586
>
> browsing is slow if use squid as a proxy server.
> on a MS Win 2003 SP2 x864, we installed a small proxy software(cc
> proxy), and then did the browsing, the result was good/excellent.
>
> please help/suggest
>
>
> details:
> --------
>
> two nics, one connected with DSL modem(eth1), and one connected with our
> local lan(eth0)
>
> runlevel script for routing and firewalling:
> --------------------------------------------
> # cat /etc/init.d/routing
>
> #!/bin/bash
>
> # /etc/init.d/routing
>
> ### BEGIN INIT INFO
> # Provides: routing
> # Required-Start: $network
> # Required-Stop:
> # Default-Start: 3 5
> # Default-Stop:
> # Description: set up routing and Firewalling
> ### END INIT INFO
>
>
> # squid server IP
> SQUID_SERVER=192.168.0.1
> # Interface connected to Internet
> INTERNET=eth1
> # Interface connected to LAN
> LAN_IN=eth0
> # Squid port
> SQUID_PORT=8080
>
> # Load IPTABLES modules for NAT and IP conntrack support
> modprobe ip_conntrack
> modprobe ip_conntrack_ftp
> # For win xp ftp client
> modprobe ip_nat_ftp
>
> . /etc/rc.status
>
> # First reset status of this service
> rc_reset
>
> case "$1" in
> stop)
> echo -n "Stoping Firewall"
> # disaable routing
> echo 0 > /proc/sys/net/ipv4/ip_forward
> # DO NOT MODIFY BELOW
> # Clean old firewall
> iptables -F
> iptables -X
> iptables -t nat -F
> iptables -t nat -X
>
> # Remember status and be verbose
> rc_status -v
> ;;
> start)
> echo -n "Starting Firewall and router"
>

Note: the netfilter guys recommend using the iptables-restore tool for
firewall setup. It's much faster and much more secure than an incremental
build of the rules like this.

> # enable routing
> echo 1 > /proc/sys/net/ipv4/ip_forward
> # Setting default filter policy
> iptables -P INPUT DROP
> iptables -P OUTPUT ACCEPT
> # Unlimited access to loop back
> iptables -A INPUT -i lo -j ACCEPT
> iptables -A OUTPUT -o lo -j ACCEPT
> # To block Msn Messenger
> iptables -A INPUT -p tcp --dport 1863 -j DROP
> iptables -A FORWARD -o "$INTERNET" -i "$LAN_IN" -d 0/0 -p tcp
> --dport 1863 -j DROP
> iptables -A FORWARD -o "$INTERNET" -i "$LAN_IN" -d 0/0 -p udp
> --dport 1863 -j DROP
> # Allow unlimited access to $INTERNET from our local lan
> iptables -A INPUT -i $INTERNET -m state --state
> ESTABLISHED,RELATED -j ACCEPT
> # set this system as a router for Rest of LAN
> iptables --table nat --append POSTROUTING --out-interface
> $INTERNET -j MASQUERADE
> iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT
> # unlimited access to LAN
> iptables -A INPUT -i $LAN_IN -j ACCEPT
> iptables -A OUTPUT -o $LAN_IN -j ACCEPT
> ### for Win2k3 SP1 virtual machine
> iptables -A INPUT -s 192.168.1.3 -m state --state
> NEW,ESTABLISHED,RELATED -j ACCEPT
> ###
> # DNAT port 80 request comming from LAN systems to squid 3128
> ($SQUID_PORT) aka transparent proxy
> iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j
> DNAT --to $SQUID_SERVER:$SQUID_PORT
> # if it is same system
> iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j
> REDIRECT --to-port $SQUID_PORT
> # DROP everything and Log it
> iptables -A INPUT -j LOG
> iptables -A INPUT -j DROP
> # Remember status and be verbose
> rc_status -v
> ;;
> restart)
> ## Stop the service and regardless of whether it was
> ## running or not, start it again.
> $0 stop
> $0 start
> # Remember status and be quiet
> rc_status
> ;;
> esac
> rc_exit
>
>
>
> Name Resolution:
> ------------------
>
> # cat /etc/resolv.conf
> nameserver 127.0.0.1
>
> # grep "forward*" /etc/named.conf
> forward first;
> forwarders { 202.163.96.3; 202.163.96.4; };
>
>
> squid.conf
> ----------
> # grep -v ^# /etc/squid/squid.conf
> acl all src all
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32
> acl to_localhost dst 127.0.0.0/8
> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
>
>
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
>
> acl CON_SKYPE method CONNECT
> acl skype url_regex ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]
> http_access allow CON_SKYPE skype
>
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
>
> ##### *****
> acl ftp_software dstdomain .sight-board.de
>
> http_access deny CONNECT !SSL_ports !ftp_software
> ##### *****
>
> http_access allow localnet
> http_access allow localhost
>

NOTE: The following rules only apply to external people attempting to
connect to your internal LAN machines.

... Or to people using your proxy as a free gateway to elsewhere on the
Internet.

They can do that to your proxy by simply sending an HTTP request to any one
of your internal LAN IPs with a forged HTTP header and URL.

> acl porn_web url_regex -i "/etc/squid/blacklists/porn/domains.txt"
> http_access deny porn_web
>
> acl hod_ip src "/etc/squid/ipes/hod_ip.txt"
> http_access allow hod_ip
>
> acl allowed_for_all url_regex -i "/etc/squid/allowed_for_all.txt"
> http_access allow allowed_for_all
>
> acl cad_ip src "/etc/squid/ipes/cad_ip.txt"
> http_access deny cad_ip
>
> acl allowed_for_env url_regex -i "/etc/squid/allowed_for_env.txt"
> http_access allow allowed_for_env
>
> acl env_ip src "/etc/squid/ipes/environment_ip.txt"
> http_access deny env_ip
>
> acl ftp_site url_regex -i ftp://ftp.sight-board.de
> http_access allow ftp_site
>
> acl serial_key url_regex -i www.phazeddl.com
> http_access deny serial_key
>
> acl ent_websites url_regex -i "/etc/squid/entwebsites.txt"
> http_access deny ent_websites
>
> acl proxyservers url_regex -i "/etc/squid/proxyservers.txt"
> http_access deny proxyservers
>
> acl web_msgrs url_regex -i "/etc/squid/webmessengers.txt"
> http_access deny web_msgrs
>
> acl blocked_extns url_regex -i "/etc/squid/blocked_extns.txt"
> http_access deny blocked_extns
>
> acl hod_tl_ip src "/etc/squid/ipes/hod_and_tl_ip.txt"
> http_access allow hod_tl_ip
>
> http_access deny all
>
> icp_access allow localnet
> icp_access deny all
>
> http_port 8080 transparent
>
> hierarchy_stoplist cgi-bin ?
>
> cache_mem 1000 MB
> maximum_object_size_in_memory 120 KB
>
> cache_dir diskd /var/cache/squid 50000 16 256

diskd is probably your problem.
From the use of iptables as a firewall I would guess that this is a linux
box. On linux you should try AUFS storage for fastest speed.
If that label is the only change on the config line you can test it with a
simple re-config.

Also with ~50GB of storage you are probably wanting to use something like
32 or 64 for the Level-1 value (currently 16). Changing that requires a
cache delete and rebuild with 'squid -z' though.

<snip>
> refresh_pattern -i \.swf$ 43200 100% 43200 override-lastmod
> override-expire
> refresh_pattern -i \.mid$ 43200 100% 43200 override-lastmod
> override-expire
> refresh_pattern -i \.wav$ 43200 100% 43200 override-lastmod
> override-expire
> refresh_pattern -i \.mp3$ 43200 100% 43200 override-lastmod
> override-expire
> refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
> refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
> refresh_pattern . 0 40% 40320

These days I'm advising people terminate their file extension patterns with
(\?.*)?$ instead of just $ to catch all the sites using dynamic parts in
their URLs.

Amos
Received on Mon Aug 24 2009 - 05:06:03 MDT