RE: Helpers idea

> -----Original Message-----
> From: dancer@zeor.simegen.com [mailto:dancer@zeor.simegen.com]
> Sent: Friday, 15 September 2000 9:55 AM
> To: Robert Collins; squid-dev@squid-cache.org
> Subject: Re: Helpers idea
>
>
> Robert Collins wrote:
>
> > I don't know if this is on the to-do list, but I'm thinking
> of coding up
> > self-adjusting helper counts.
> >
> > something along the lines of
> >
> > n is the number of running helper of a given type
> > when new jobs are submitted, if the queue length for n is
> more than 2 * n,
> > and no helpers have been spawned for n/2 jobs spawn n/10
> (rounded up) new
> > helpers.
> > if the queue length stays at zero for n/2 jobs kill n/10 helpers.
> >
> > Yes I know magic numbers are bad - I'll have them in the conf..
> >
> > Comments/already done/bad idea?
> >
> > Rob
>
> I'm just experimenting with skipping accept() if the queue
> length is over a
> arbitrary number.
>
> D
>
>

I presume you mean you do stop accepting new http requests?
There are two essential problems with that
1. Existing persistent connections can still make requests. In the event
of a request, a helper may be called (i.e. a DNS lookup may be needed
(->so the dns helper gets called). (And yes I know about internal dns
:-]). So a fixed limit on helpers is still problematical.
2. If you don't accept() then the user may time out if the (whatever
helper queue you measure on) queue length doesn't drop fast enough.
Network problems that affect one particular helper may not affect all
helpers, so I think it is unreasonable to affect all users. (DNS is a
good example here. If your dns server is down, new name lookups will
fail, and a queue may build up. But http should still be working fine.
Redirectors are also a good example. What if all but one redirector
process has hung? Redirections are still happening, but one will never
complete, and the queue will be getting pretty long.

I think a better approach than skipping accept() would be stop listening
in the event that everything has crashed and burnt.. that way there will
be no time out, the users machine will know immediately they can't get
thru and their .pac script can switch proxies (or whatever their fall
back is - ring the admin etc etc)

Rob
Received on Thu Sep 14 2000 - 18:41:03 MDT