Re: [squid-users] Squid box dropping connections

On 17/11/2011 16:11, Nataniel Klug wrote:
> Hello all,
>
> I am facing a very difficult problem in my network. I am
> using a layout like this:
>
> (internet) ==<router> ==<squid> == [clients]
>
> I am running CentOS v5.1 with Squid-2.6 STABLE22 and Tproxy
> (cttproxy-2.6.18-2.0.6). My kernel is kernel-2.6.18-92. This is the most
> reliable setup I ever made running Squid. My problem is that I am having
> serious connections troubles when running squid over 155000 conntrack
> connections.
>
> From my clients I start losing packets to router when the
> connections go over 155000. My kernel is prepared to run over 260k
> connections. I am sending a screenshot about the problem where I have 156k
> connections and I started connections on port 80 to go through squid (bellow
> I will post every rule I am using for my firewall and transparent
> connections, also I will send my squid.conf).
>
> http://imageshack.us/photo/my-images/12/problemsg.png/
>
> The configuration I am using:
>
> /etc/firewall/firewall
> #!/bin/bash
> IPT="/sbin/iptables"
> RT="/sbin/route"
> SYS="/sbin/sysctl -w"
> $IPT -F
> $IPT -t nat -F
> $IPT -t nat -X
> $IPT -t mangle -F
> $IPT -t mangle -X
> $IPT -t filter -F
> $IPT -t filter -X
> $IPT -X
> $IPT -F INPUT
> $IPT -F FORWARD
> $IPT -F OUTPUT
> $SYS net.ipv4.ip_forward=1
> $SYS net.ipv4.ip_nonlocal_bind=1
> $SYS net.ipv4.netfilter.ip_conntrack_max=262144
>
> /etc/firewall/squid-start
> #!/bin/bash
> IP="/sbin/ip"
> IPT="/sbin/iptables"
> FWDIR="/etc/firewall"
> /etc/firewall/firewall
> $IPT -t tproxy -F
> for i in `cat $FWDIR/squid-no-dst`
> do
> $IPT -t tproxy -A PREROUTING -d $i -j ACCEPT
> done
> for i in `cat $FWDIR/squid-no-src`
> do
> $IPT -t tproxy -A PREROUTING -s $i -j ACCEPT
> done
> $IPT -t tproxy -A PREROUTING -p tcp --dport 80 -j TPROXY --on-port 3128
>
> /etc/squid/squid.conf
> http_port 3128 tproxy transparent
> tcp_outgoing_address XXX.XXX.144.67
> icp_port 0
>
> cache_mem 128 MB
>
> cache_swap_low 92
> cache_swap_high 96
> maximum_object_size 1000000 KB
> cache_replacement_policy heap LFUDA
> memory_replacement_policy heap LFUDA
>
> cache_dir aufs /cache/01/01 47000 64 256
> cache_dir aufs /cache/01/02 47000 64 256
> cache_dir aufs /cache/02/01 47000 64 256
> cache_dir aufs /cache/02/02 47000 64 256
> cache_dir aufs /cache/03/01 47000 64 256
> cache_dir aufs /cache/03/02 47000 64 256
> #--[ Max Usage : by Drive ]--#
> # sdb1 ( max = 228352 / usg = 95400 (41,77%) ]
> # sdb1 ( max = 228352 / usg = 95400 (41,77%) ]
> # sdb3 [ max = 234496 / usg = 95400 (40,68%) ]
> #-- [ Max HDD sdb Usage ]--#
> # sdb [ max = 923994 / aloc = 691200 (74,81%) ]
>
> cache_store_log none
> access_log /usr/local/squid/var/logs/access.log squid
> client_netmask 255.255.255.255
> ftp_user squid_at_cnett.com.br
>
> diskd_program /usr/local/squid/libexec/diskd
> unlinkd_program /usr/local/squid/libexec/unlinkd
>
> error_directory /usr/local/squid/share/errors/Portuguese
>
> dns_nameservers XXX.XXX.144.14 XXX.XXX.144.6
>
> acl all src 0.0.0.0/0
> acl localhost src 127.0.0.1/32
> acl to_localhost dst 127.0.0.0/8
> acl QUERY urlpath_regex cgi-bin \?
> acl SSL_ports port 443
> acl Safe_ports port 80 21 443 70 210 280 488 591 777 1025-65535
> acl CONNECT method CONNECT
>
> acl ASN53226_001 src XXX.XXX.144.0/22
> acl ASN53226_002 src XXX.XXX.148.0/22
>
> http_access allow ASN53226_001
> http_access allow ASN53226_002
>
> http_access allow localhost
> http_access allow to_localhost
>
> cache deny QUERY
>
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access deny all
> icp_access deny all
>
> cache_mgr suporte_at_cnett.com.br
> cache_effective_user squid
> cache_effective_group squid
> visible_hostname cache
> unique_hostname 02.cache
>
> When I first start linux and there is just a few connections
> going through the squid box it works just fine. When the connections go over
> 155k the problems began. Is there anything I can do to solve the problem?
well this is one of the big problems of the conntrack thingy..
what you can try is to also to change the tcp to:
sysctl net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=3600
cause it might causing the problem of such a huge ammount of connection
tracking size.
the basic size is 120 minutes which can cause a lot of troubles in many
cases of open connections.
and by the way.. do you really have 155K connections? it seems like too
much.

hope to hear more about the situation.

Regards Eliezer
>
> --
> Att,
>
> Nataniel Klug
>
>
Received on Thu Nov 17 2011 - 23:11:16 MST