Re: Squid connection retry problem -- fd leak?

On Fri, 12 Sep 1997, Michael Pelletier wrote:

> On Fri, 12 Sep 1997, Apiset Tananchai wrote:
> > Mike,
> > A couple of days ago I just patch my squid NOVM 1.1.14 with your
> > connection retry patch and found that my squid start to constantly consume
> > more and more fd. Currently I use Oscar & Michael 3000fd patch to increase
> > number of fd for squid so it's not bomb yet. :)
>
> What kind of connection load do you see? My largest FD in use is 43, and
> there's none in use betwen 28 and 43, and I get about 1000 - 2000
> connections per hour. Of course, it's early morning here and it
> log-rotated last night.

Hm...our server is getting ~20000 connection/hour. I used to get around
30000 connection/hour during peak load. Output from netstat shows that our
server had around 200-300 tcp connections during that time. As I told you
below, most fds were hanging in writing state.

> > It maybe because I'm rotating squid logfile at midnight. The fd usage drop
> > dramatically (from around 1900 to 250) after that. Please visit
> > http://demo.ksc.co.th/stat/1-week.html and click on Filedescriptor Usage
> > link to get a graph of what I'm talking. I'm switching to a new patch on
> > 9/11 and you'll notice constantly increase of fd after that.
> >
> > I used cachemgr.cgi to check fd usage and found that most fds are file fd
> > which were opened for writing. Maybe 5-10 times more than before patch.
> >
> > Is this behavior is normal? I know that you also use this patch for your
> > squid too. What platform are you using? Follow is mine:
>
> No, I definitely don't think it's normal. That graph is pretty dramatic.
> I'm not running the NOVM version, though. I really need to get MRTG
> running on my Squid, perhaps I'm having this problem and just don't know
> it.
>
> I suppose it might have something to do with the adjustments being made to
> the FD lifetime. Or maybe the descriptors being retried aren't being
> recycled by the comm_reset_fd() function call...

The situation is even worse for today. Even we had rotated the log file on
09/13 midnight but the fd usage is still going above 2200 and the server
stop responding to all connection since around last midnight(09/14, please
check the graph). By the way, I can telnet to the server and restart
squid. I didn't notice any delay when telnet to a server.

I already backout connection-retry patch and only running 1.NOVM.14 with
fixed for icp.c and unlinkd.c and it seems to work fine for the moment.

Does anyone using connection retry patch with 1.NOVM.14 got similar
problem with me? And do you have any suggestion to fix this problem?

One more question, Mike. :) Does your patch solve the problem with
"connection reset by peer" error I usually get when browsing microsoft
homepage or pages that use IIS as a server? I usually get complete page
after press backward & forward button.

Thanks for you help

--
aet
.x.