Re: [squid-users] Increasing File Descriptors - Fixed!!

Hi,

Thanks for the tip!  I'm also aware of the "conntrack" option with
iptables.  But, we prefer not to run any firewall on our Squid boxes.
We run firewall at the NOC edges.

I was thinking in the line that, Squid is already aware what outbound
connections it is initiating and closing, so it might be useful to
have an internal counter do that.

Regards
HASSAN

On Thu, May 13, 2010 at 20:41, Jose Ildefonso Camargo Tolosa
<ildefonso.camargo_at_gmail.com> wrote:
>
> Hi!
>
> There are a couple of tricks to get this, at least on Linux systems.
>
> 1. I used to use a fancy trick for this: conntrack.
>
> cat /proc/net/ip_conntrack
>
> With conntrack module loaded, usually, just add a few iptables rules,
> maybe just as simple as:
>
> iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>
> That should do the trick, then on the file you get something like:
>
> cat /proc/net/ip_conntrack
>
> tcp      6 431989 ESTABLISHED src=10.16.0.73 dst=74.125.45.83
> sport=50048 dport=443 packets=11 bytes=8765 src=74.125.45.83
> dst=10.16.0.73 sport=443 dport=50048 packets=9 bytes=2218 [ASSURED]
> mark=0 secmark=0 use=2
> tcp      6 431997 ESTABLISHED src=10.16.0.73 dst=74.125.67.17
> sport=49949 dport=443 packets=15 bytes=10729 src=74.125.67.17
> dst=10.16.0.73 sport=443 dport=49949 packets=15 bytes=2785 [ASSURED]
> mark=0 secmark=0 use=2
> tcp      6 431999 ESTABLISHED src=95.211.91.20 dst=10.16.0.73
> sport=1723 dport=43648 packets=13385 bytes=19006700 src=10.16.0.73
> dst=95.211.91.20 sport=43648 dport=1723 packets=6446 bytes=257840
> [ASSURED] mark=0 secmark=0 use=2
> tcp      6 431987 ESTABLISHED src=10.16.0.73 dst=74.125.45.83
> sport=34690 dport=443 packets=11 bytes=3759 src=74.125.45.83
> dst=10.16.0.73 sport=443 dport=34690 packets=9 bytes=1903 [ASSURED]
> mark=0 secmark=0 use=2
> tcp      6 431987 ESTABLISHED src=10.16.0.73 dst=74.125.45.83
> sport=34692 dport=443 packets=25 bytes=11174 src=74.125.45.83
> dst=10.16.0.73 sport=443 dport=34692 packets=23 bytes=15733 [ASSURED]
> mark=0 secmark=0 use=2
>
> As you can see, from there you can extract the source IP and source
> port,and other info, luke the connection state, the connection or sec
> mark, packets and bytes associated with the connection, that info can
> be useful.
>
> This will help to know which local ports are being used on active connections.
>
> 2. Another, maybe simpler, way:
>
> netstat -na
>
> That gives something like this:
>
> tcp        0      0 10.16.0.73:34692        74.125.45.83:443        ESTABLISHED
> tcp        0      0 10.16.0.73:34690        74.125.45.83:443        TIME_WAIT
> tcp        0      0 10.16.0.73:49949        74.125.67.17:443        ESTABLISHED
> tcp        0      0 10.16.0.73:50048        74.125.45.83:443        ESTABLISHED
> tcp        0      0 10.16.0.73:43648        95.211.91.20:1723       ESTABLISHED
>
> This doesn't give bytes nor packets, but enough to get the ports
> involved on the connection.
>
> I hope this helps,
>
> Ildefonso Camargo
>
> On Thu, May 13, 2010 at 8:32 AM, Nyamul Hassan <mnhassan_at_usa.net> wrote:
> > Is it possible to have a CacheMgr parameter that shows the total
> > number of ports that are being used by Squid?
> >
> > That might be complicated though.  We use different IP addresses for
> > outgoing requests, so number of ports would make sense if available on
> > a IP wise breakdown.
> >
> > Regards
> > HASSAN
> >
> >
> > 2010/5/13 Bradley, Stephen W. Mr. <bradlesw_at_muohio.edu>
> >>
> >> I wish I had snapshots of our stats when we were up and under full load so I could give you and idea of how it was working.
> >>
> >> But it was working well right up until the ragged edge (and my FD limit at the time).
> >>
> >> steve
> >>
> >> -----Original Message-----
> >> From: Henrik Nordström [mailto:henrik_at_henriknordstrom.net]
> >> Sent: Wednesday, May 12, 2010 3:57 PM
> >> To: Bradley, Stephen W. Mr.
> >> Cc: Squid Users
> >> Subject: RE: [squid-users] Increasing File Descriptors - Fixed!!
> >>
> >> ons 2010-05-12 klockan 10:17 -0400 skrev Bradley, Stephen W. Mr.:
> >> > I went ahead and updated /etc/sysctl.conf to allow for 1024-64000 ports open for connections.
> >> >
> >> > That should help; I just wish I could test it before August.........  :-(
> >>
> >> Reminds me that we should implement some smart address + port selection
> >> algorithm. The default port selection algorithm provided by the
> >> operating systems is very stupid and quickly runs out of free ports even
> >> if you increase the range to max possible. A maximized range of 60K
> >> ports can sustain only about 500 connections / second.
> >>
> >> Thankfully that's quite sufficient for most Squid setups still, but not
> >> all, and with improvements in multi-core support it will become much
> >> more of an issue.
> >>
> >> Regards
> >> Henrik
> >>
> >>
> >>
> >
>
Received on Thu May 13 2010 - 23:56:13 MDT