Re: [squid-users] Too many queued redirector requests

Hi Hendrik,
    I have been following this thread of mails and I
have a problem with the number of redirectors too.
    I am using squid-2.5.STABLE5 configured to work
with caching disabled. I am using a redirector that
does some url filtering using a local database. I have
timed the redirector - it takes anywhere between 42
msecs to 2000 msecs at times to process. I am running
squid on a system with the configuration -
Linux-2.4.18-14, P4, 512 MB RAM. We have other
applications like a web server, etc that are also
running on this system.
    You had mentioned in ur earlier mail that ur
system is so configured that 80% of the requests are
handled by the first redirector, 10% by the second and
so on. Could you kindly elaborate as to how this
done - or is it the way squid works? If this is the
case, then adding more redirectors shd not solve my
problem.
    I tried conducting some simple load tests with
squid using the redirectors.
   1 redirector worked fine for 5 simultaneous browser
clients(that is w/o throwing a FATAL error and
restarting),
   2 redirectors worked fine for 14 browser clients
  but subsequent tests showed that even with 5
redirector clients,20 browser clients cud not be
handled simultaneously. I don't want to enable
redirctor bypass though.
   I am failing to understand this behaviour. I would
be thankful if u cud spare some time to explain what
cud be happening here and tell me a solution for it.
   Regards and TIA,
       Deepa
   

 --- Hendrik_Voigtländer <hendrik@voigtlaenders.net>
wrote: > Hi,
>
> E250 with how many processor of what type?
> Probably you have an performance problem with the
> sleepycat berkeley-db.
> If all your processors are busy all the time
> increasing the number of
> redirectors won't help.
>
> In this case I would switch over to Linux with an
> Intel machine. We have
> replaced our old E450 with an HP/Compaq ML370
> (decent machine, but not
> high end) with a significant improve in squid
> perfomance. I gave up on
> compiling squidguard on solaris, to much hassle and
> to much load
> probably as well for the 168MHz(!) processors.
> With debian no problem at all, 'apt-get install
> squidguard' and done...
>
> I really like the idea of using multiple cheap
> machine with
> loadbalancing and failover, but IMHO you need to use
> automatic proxy
> configuration to achieve this. I would use server
> hardware for this, but
> something cheaper than HP/Compaq, for instance
> Supermicro.
>
> Get cachemgr.cgi running, it is really useful to
> look at squid &
> squidguards status.
>
> # TAG: redirector_bypass
> # When this is 'on', a request will not go
> through the
> # redirector if all redirectors are busy. If
> this is 'off'
> # and the redirector queue grows too large,
> Squid will exit
> # with a FATAL error and ask you to increase
> the number of
> # redirectors. You should only enable this if
> the redirectors
> # are not critical to your caching system. If
> you use
> # redirectors for access control, and you
> enable this option,
> # then users may have access to pages that
> they should not
> # be allowed to request.
> #
> redirector_bypass on
>
> As you may have noticed it is impossible to filter
> 100% of all unwanted
> stuff, bypassing in high load situations won't make
> things much worse.
>
> Keep an eye on the redirector stats in cachemgr how
> many requests are
> actually bypassing squidguard. In our setup it is
> less than 1%.
>
> Regards, Hendrik.
>
>
> Merid Tilahun wrote:
> > Thanx Hendrik
> > I am running squid on solaris 8, sun enterprise
> 250
> > machine. I have more that 500 users connect at
> peak
> > hour.
> > I never got around to configure cachemanager.cgi,
> I
> > will look in to that.
> > I use squidguard to filter porn, and it seems to
> be
> > working but it is affecting the servicetime badly.
> I
> > run around 20 redirector processes, I have been
> > increasing constantly.
> > I deactivated squidguard for a while and I got not
> > messages, but I need squidGuard to block the porn.
> > What is redirector bypass and how do i enable it?
> >
> > --- Hendrik Voigtlaender
> <hendrik@voigtlaenders.net>
> > wrote:
> >
> >>Hi,
> >>
> >>I just checked my logs:
> >>We get 40...50req/sec for about 10 hours a day
> with
> >>nearly no traffic
> >>during the night, this levels out to a daily
> average
> >>of 1000req/min.
> >>Number of users are reported with about
> 1000...1500,
> >>but I guess that we
> >>have never more than 150 clients connect at the
> same
> >>time.
> >>We have 15 redirectors running, average redirector
> >>service time is not
> >>measurable, servicetimes are good. In cachemgr.cgi
> >>you can check how
> >>many request are handled by each redirector.
> >>In our setup, more than 80% of the requests are
> >>handled by the first
> >>one. The second gets about 10%, all the other are
> >>used only during peak
> >>periods. Redirectors #15 is idle nearly the whole
> >>time.
> >>In top only a a few (2...3) squidguards are
> showing
> >>enough activity to
> >>be noticed, but not much. Machine is Intel 2,4GHz,
> >>2x36GB SCSI
> >>cache_dir, 2GB RAM, Linux debian woody, squid &
> >>squidguard out of distro.
> >>
> >>I played around with the number of redirectors but
> >>never got rid of this
> >>message (all busy, increase blabla). I think it is
> >>not critical if this
> >>warning appears on and off, but not to often. 15
> is
> >>IMHO more than
> >>sufficient with this load.
> >>
> >>What are you using squidguard for? If you try to
> >>block pr0n, you can as
> >>well enable redirector bypass. The blacklist will
> >>never catch all sites
> >>anyway, it doesnt matter if a couple of more
> request
> >>bypass squidguard.
> >>This will eliminate the warning for sure.
> >>
> >>How many redirectors are you using ? I would not
> >>install squidguard but
> >>probably deativate it in squid.conf for testing.
> >>I can imagine that too many redirectors are eating
> >>up the ressources
> >>badly needed by squid, especially on a slow/small
> >>machine.
> >>Can you post some specs?
> >>
> >>Regards, Hendrik Voigtländer
> >>
> >>Merid Tilahun wrote:
> >>
> >>
> >>>Hi all:
> >>>I have istalled and configured squid and it was
> >>>working
> >>>fine until now. I installed squidGuard and when
> >>
> >>squid
> >>
> >>>starts working with it I get the user warning
> >>
> >>messages
> >>
> >>>Too many queued redirector requests. I have
> >>
> >>increased
> >>
> >>>the number of redirector processes but I am still
> >>>getting this messages. Plus the retrieval time
> for
> >>>sites is too long. Does any one know a way out?
> or
> >>>shall I just
> >>>uninstall squidguard?
> >>>My squid get around 850 req/min ( got this from
> >>
> >>MRTG).
> >>
> >>>Thank you in advance
> >>>Regards
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>__________________________________
> >>>Do you Yahoo!?
> >>>Friends. Fun. Try the all-new Yahoo! Messenger.
> >>>http://messenger.yahoo.com/
> >>>
> >>>
> >>
>
=== message truncated ===

________________________________________________________________________
Yahoo! India Matrimony: Find your partner online. http://yahoo.shaadi.com/india-matrimony/
Received on Sun May 30 2004 - 22:50:17 MDT