Re: [squid-users] Too many queued redirector requests

Hi,
   Thanks for the response.
   Cache takes a considreable amt of memory which our
system cannot afford, hence the choice of disbling the
cache.
   The redirector that I am using is a self made one
for content filtering. It communicates with a remote
server to obtain the details of the content of the
url. This communication alone could take a max of
2000msecs.
I cannot avoid this though. What is the average time
for a redirector?
   What is the maximum queue length for the
redirectors ? Is this a configurable parameter? I have
seen FATAL errors being thrown for 22 on 10
redirectors. Isn't this a very small number?
   Plz tell me if I can change this parameter some
place and correct the problem.
   Regards and TIA,
      Deepa

--- Hendrik_Voigtländer <hendrik@voigtlaenders.net>
wrote: > Hi,
>
> > I have been following this thread of mails and
> I
> > have a problem with the number of redirectors too.
> > I am using squid-2.5.STABLE5 configured to
> work
> > with caching disabled.
> Just curious: Any reason to disable caching?
>
> > I am using a redirector that
> > does some url filtering using a local database.
>
> What is the purpose? Filtering offending content?
> Selfmade redirector? What kind of database?
>
> > I have
> > timed the redirector - it takes anywhere between
> 42
> > msecs to 2000 msecs at times to process. I am
> running
> > squid on a system with the configuration -
> > Linux-2.4.18-14, P4, 512 MB RAM. We have other
> > applications like a web server, etc that are also
> > running on this system.
>
> 2000msecs? Thats a lot...
>
> > You had mentioned in ur earlier mail that ur
> > system is so configured that 80% of the requests
> are
> > handled by the first redirector, 10% by the second
> and
> > so on. Could you kindly elaborate as to how
> this
> > done - or is it the way squid works? If this is
> the
> > case, then adding more redirectors shd not solve
> my
> > problem.
>
> The number of requests handled by each redirector as
> described in the
> posts is more a phenomenon, not a configuration
> issue.
> - Squidguard is very fast and IMHO the request are
> not distributed round
> robin, but to the first idle redirector
> - light load - the first redirector is always ready
> to serve the
> request, all the others have never a chance to
> answer.
> - increased load - sometime the first redirector is
> busy, some requests
> are answered by the 2nd redirector
> - even higher load - sometime both 1st and 2nd
> redirectors are busy,
> the 3th gets a chance to do some work - and so on
> with the others if
> load increases.
> - load peak - all redirectors get something to do,
> some request are
> queued and the warning message occurs, but never a
> FATAL error as the
> queue lenght never gets high (long?) enough.
>
> I think your problem is different as your
> redirectors are quite slow,
> the queues are growing to long which causes the
> fatal error.
> Increasing the number of redirectors should help a
> bit in your case, but
> probably the database is to slow to handle all the
> requests. If the
> database is the problem, increasing the no of
> redirector means
> increasing the processing time as well - this means
> the number of
> request which can be processed in a given amount of
> time will not
> increase much.
>
> > I tried conducting some simple load tests with
> > squid using the redirectors.
> > 1 redirector worked fine for 5 simultaneous
> browser
> > clients(that is w/o throwing a FATAL error and
> > restarting),
> > 2 redirectors worked fine for 14 browser
> clients
> > but subsequent tests showed that even with 5
> > redirector clients,20 browser clients cud not be
> > handled simultaneously. I don't want to enable
> > redirctor bypass though.
> > I am failing to understand this behaviour. I
> would
> > be thankful if u cud spare some time to explain
> what
> > cud be happening here and tell me a solution for
> it.
> > Regards and TIA,
> > Deepa
> >
> >
> > --- Hendrik_Voigtländer
> <hendrik@voigtlaenders.net>
> > wrote: > Hi,
> >
> >>E250 with how many processor of what type?
> >>Probably you have an performance problem with the
> >>sleepycat berkeley-db.
> >>If all your processors are busy all the time
> >>increasing the number of
> >>redirectors won't help.
> >>
> >>In this case I would switch over to Linux with an
> >>Intel machine. We have
> >>replaced our old E450 with an HP/Compaq ML370
> >>(decent machine, but not
> >>high end) with a significant improve in squid
> >>perfomance. I gave up on
> >>compiling squidguard on solaris, to much hassle
> and
> >>to much load
> >>probably as well for the 168MHz(!) processors.
> >>With debian no problem at all, 'apt-get install
> >>squidguard' and done...
> >>
> >>I really like the idea of using multiple cheap
> >>machine with
> >>loadbalancing and failover, but IMHO you need to
> use
> >>automatic proxy
> >>configuration to achieve this. I would use server
> >>hardware for this, but
> >>something cheaper than HP/Compaq, for instance
> >>Supermicro.
> >>
> >>Get cachemgr.cgi running, it is really useful to
> >>look at squid &
> >>squidguards status.
> >>
> >># TAG: redirector_bypass
> >># When this is 'on', a request will not go
> >>through the
> >># redirector if all redirectors are busy.
> If
> >>this is 'off'
> >># and the redirector queue grows too large,
> >>Squid will exit
> >># with a FATAL error and ask you to increase
> >>the number of
> >># redirectors. You should only enable this
> if
> >>the redirectors
> >># are not critical to your caching system.
> If
> >>you use
> >># redirectors for access control, and you
> >>enable this option,
> >># then users may have access to pages that
> >>they should not
> >># be allowed to request.
> >>#
> >>redirector_bypass on
> >>
> >>As you may have noticed it is impossible to filter
> >>100% of all unwanted
> >>stuff, bypassing in high load situations won't
> make
> >>things much worse.
> >>
> >>Keep an eye on the redirector stats in cachemgr
> how
> >>many requests are
> >>actually bypassing squidguard. In our setup it is
> >>less than 1%.
> >>
> >>Regards, Hendrik.
> >>
> >>
> >>Merid Tilahun wrote:
> >>
> >>>Thanx Hendrik
> >>>I am running squid on solaris 8, sun enterprise
> >>
> >>250
> >>
> >>>machine. I have more that 500 users connect at
> >>
> >>peak
> >>
> >>>hour.
> >>>I never got around to configure cachemanager.cgi,
> >>
> >>I
> >>
> >>>will look in to that.
> >>>I use squidguard to filter porn, and it seems to
> >>
> >>be
>
=== message truncated ===

________________________________________________________________________
Yahoo! India Matrimony: Find your partner online. http://yahoo.shaadi.com/india-matrimony/
Received on Mon May 31 2004 - 05:52:03 MDT