Re: [Fwd: Re: [squid-users] help with squid redirectors]

 On Tue, 08 Mar 2011 07:52:50 -0500, Osmany wrote:
> -------- Forwarded Message --------
> From: Osmany <osmany_at_oc.quimefa.cu>
> Reply-to: osmany_at_oc.quimefa.cu
> To: squid-users <squid-users_at_squid-cache.org>
> Subject: Re: [squid-users] help with squid redirectors
> Date: Tue, 08 Mar 2011 07:41:47 -0500
>
> On Wed, 2011-03-09 at 01:33 +1300, Amos Jeffries wrote:
>> On 09/03/11 01:20, Osmany wrote:
>> > On Tue, 2011-03-08 at 12:21 +1300, Amos Jeffries wrote:
>> >> On Tue, 08 Mar 2011 11:58:57 +1300, Amos Jeffries wrote:
>> >>> On Mon, 07 Mar 2011 16:59:07 -0500, Osmany wrote:
>> >>>> Greetings everyone,
>> >>>>
>> >>>> So I'm having trouble with my squid proxy-cache server. I
>> recently
>> >>>> added
>> >>>> a redirect program because I had to make users go to my
>> kaspersky
>> >>>> admin
>> >>>> kit and my WSUS services to get their updates and it works fine
>> but
>> >>>> I
>> >>>> get constantly a warning and squid just collapses after a few
>> >>>> minutes of
>> >>>> run time. This is what I get in my cache.log:
>> >>>>
>> >>>> 2011/03/07 15:54:17| WARNING: All url_rewriter processes are
>> busy.
>> >>>> 2011/03/07 15:54:17| WARNING: up to 465 pending requests queued
>> >>>> 2011/03/07 15:54:17| storeDirWriteCleanLogs: Starting...
>> >>>> 2011/03/07 15:54:17| WARNING: Closing open FD 1455
>> >>>> 2011/03/07 15:54:17| commSetEvents: epoll_ctl(EPOLL_CTL_DEL):
>> failed
>> >>>> on
>> >>>> fd=1455: (1) Operation not permitted
>> >>>> 2011/03/07 15:54:17| 65536 entries written so far.
>> >>>> 2011/03/07 15:54:17| 131072 entries written so far.
>> >>>> 2011/03/07 15:54:17| WARNING: Closing open FD 1456
>> >>>> 2011/03/07 15:54:17| commSetEvents: epoll_ctl(EPOLL_CTL_DEL):
>> failed
>> >>>> on
>> >>>> fd=1456: (1) Operation not permitted
>> >>>> 2011/03/07 15:54:17| Finished. Wrote 139965 entries.
>> >>>> 2011/03/07 15:54:17| Took 0.1 seconds (1288729.1
>> entries/sec).
>> >>>> FATAL: Too many queued url_rewriter requests (465 on 228)
>> >>>> Squid Cache (Version 2.7.STABLE7): Terminated abnormally.
>> >>>>
>> >>>> This is what I have in the squid.conf
>> >>>>
>> >>>> # TAG: url_rewrite_program
>> >>>> url_rewrite_program /etc/squid/redirect
>> >>>>
>> >>>> # TAG: url_rewrite_children
>> >>>> url_rewrite_children 100
>> >>>>
>> >>>> # TAG: url_rewrite_concurrency
>> >>>> url_rewrite_concurrency 50
>> >>>>
>> >>>> # TAG: url_rewrite_access
>> >>>> url_rewrite_access allow redirect
>> >>>>
>> >>>> And this is what I have in my redirector script
>> >>>>
>> >>>> #!/usr/bin/perl
>> >>>> BEGIN {$|=1}
>> >>>> while (<>) {
>> >>>> @X = split;
>> >>>> $url = $X[0];
>> >>>> if ($url =~ /^http:\/\/dnl(.*)kaspersky(.*)com(.*)/) {
>> >>>> print
>> >>>> "301:ftp:\/\/dnl-kaspersky\.quimefa\.cu\:2122\/Updates";
>> >>>> }
>> >>>> elsif ($url =~ /^http:\/\/(.*)windowsupdate(.*)/) {
>> >>>> print
>> "301:http:\/\/windowsupdate\.quimefa\.cu\:8530";
>> >>>> }
>> >>>> }
>> >>>>
>> >>>> Can you please help me to solve this?
>> >>>
>> >>> Your script does not support concurrency. When that is
>> configured in
>> >>> squid there will be 2 space-delimited fields to handle.
>> >>> First one being the ID of the request channel, not the URL.
>> >>>
>> >>
>> >> Oops, I missed a few other things too:
>> >> * 'else' case is needed to print the no-change result back to
>> Squid
>> >> * newlines need to be printed in perl
>> >>
>> >>
>> >> $url = $X[1];
>> >> if ($url =~ /^http:\/\/dnl(.*)kaspersky(.*)com(.*)/) {
>> >> print $X[0]."
>> >> 301:ftp:\/\/dnl-kaspersky\.quimefa\.cu\:2122\/Updates\n";
>> >> }
>> >> elsif ($url =~ /^http:\/\/(.*)windowsupdate(.*)/) {
>> >> print $X[0]."
>> 301:http:\/\/windowsupdate\.quimefa\.cu\:8530\n";
>> >> }
>> >> else {
>> >> print $X[0]."\n";
>> >> }
>> >>
>> >> Amos
>> >
>> > So this is what I have now but it doesn't work. I've tried it
>> manually:
>> >
>> > #!/usr/bin/perl
>> > BEGIN {$|=1}
>> > while (<>) {
>> > @X = split;
>> > $url = $X[1];
>> > if ($url =~ /^http:\/\/dnl(.*)kaspersky(.*)com(.*)/) {
>> > print $X[0]."
>> > 301:ftp:\/\/dnl-kaspersky\.quimefa\.cu\:2122\/Updates\n";
>> > }
>> > elsif ($url =~ /^http:\/\/(.*)windowsupdate(.*)/) {
>> > print $X[0]."
>> 301:http:\/\/windowsupdate\.quimefa\.cu\:8530\n";
>> > }
>> > else {
>> > print $X[0]."\n";
>> > }
>> > }
>> >
>> > it just keeps on returning the same url that I enter. help please?
>> >
>>
>> Did you add the concurrency channel ID before the URL on each
>> manually
>> entered line?
>> eg $id $url $garbage
>>
>> Amos
>
> Yes you are right. I was missing the concurrency channel ID on my
> manual
> test in fact when I supervise my access.log I see it is actually
> working
> and when I look at the cache.log I see that it working concurrently
> with
> all the available channels.
>
> Now actually I found out that my script only works for one of the two
> conditions I have. the one that works: windowsupdate, I can see that
> it
> is actually redirecting because I see it in the access.log file like
> this:
>
> TCP_MISS/301 330 HEAD
> http://download.microsoft.com/windowsupdate/v5/redir/wuredir.cab?
>
> But the other condition, the kaspersky one I am pretty sure it is not
> working because in the access.log this is what I see:
> TCP_MISS/200 946 GET
>
> http://dnl-17.geo.kaspersky.com/diffs/bases/av/kdb/i386/base048c.kdc.hax
>
> Not only that. I can also see in the cache.log that it is giving me a
> warning that I really don't understand.
>
> helperHandleRead: unexpected reply on channel 3301 from url_rewriter
> #26
> '3301:ftp://dnl-kaspersky.quimefa.cu:2122/Updates'
>
> could you please help me with this now?

 The line which wrapped in the emails should have a space between the "
 and 3.

 ie: print $X[0]." 301:

 Amos
Received on Wed Mar 09 2011 - 02:22:28 MST