Re: [squid-users] Squid Pauses!

On Aug 27, 2013, at 10:15 AM, Golden Shadow <firas73737_at_yahoo.com> wrote:

> Hi Guy,
>
> Thanks for your reply.
>
> The guys who monitor the server say that the problem happened several times today but I can see in /var/log/messages just few log entries about squid coredumps, which were deleted for being corrupted! Today, I had one chance to inspect the problem when it happened and there was a coredump.
>
> Aug 27 12:00:58 proxy2 kernel: squid[13009]: segfault at e3d ip 000000000064d097 sp 00007fff1c23c5d0 error 6 in squid[400000+3d4000]
> Aug 27 12:05:33 proxy2 squid[6917]: Squid Parent: will start 1 kids
> Aug 27 12:05:33 proxy2 squid[6917]: Squid Parent: (squid-1) process 6920 started
> Aug 27 12:13:06 proxy2 abrt[6794]: Saved core dump of pid 13009 (/usr/sbin/squid) to /var/spool/abrt/ccpp-2013-08-27-12:00:58-13009 (5
> 1502256128 bytes)
> Aug 27 12:13:06 proxy2 abrtd: Directory 'ccpp-2013-08-27-12:00:58-13009' creation detected
> Aug 27 12:13:06 proxy2 squid[13006]: Squid Parent: (squid-1) process 13009 exited due to signal 11 with status 0
> Aug 27 12:13:22 proxy2 abrtd: Sending an email...
> Aug 27 12:13:23 proxy2 abrtd: send-mail: fatal: parameter inet_interfaces: no local interface found for ::1
> Aug 27 12:13:23 proxy2 abrtd: Error running '/bin/mailx'
> Aug 27 12:13:23 proxy2 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2013-08-27-12:00:58-13009' exited with 1
> Aug 27 12:13:23 proxy2 abrtd: Corrupted or bad directory '/var/spool/abrt/ccpp-2013-08-27-12:00:58-13009', deleting
> Aug 27 12:22:14 proxy2 squid[6917]: Squid Parent: (squid-1) process 6920 exited with status 0
> Aug 27 12:23:44 proxy2 squid[7281]: Squid Parent: will start 1 kids
> Aug 27 12:23:44 proxy2 squid[7281]: Squid Parent: (squid-1) process 7284 started
>
> Best regards,
> Firas

OK, then the pauses in service by squid are due to the amount of time required to dump core to disk for such a large process.

Please follow the instructions at http://wiki.squid-cache.org/SquidFaq/BugReporting to assist diagnosing the cause of the signal 11.

Hope this helps,
Guy

>
>
> ----- Original Message -----
> From: Guy Helmer <guy.helmer_at_palisadesystems.com>
> To: Golden Shadow <firas73737_at_yahoo.com>
> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
> Sent: Tuesday, August 27, 2013 3:46 PM
> Subject: Re: [squid-users] Squid Pauses!
>
> On Aug 27, 2013, at 6:11 AM, Golden Shadow <firas73737_at_yahoo.com> wrote:
>
>> Hello Squid Users!
>>
>> I have a TPROXY server built with squid V. 3.3.7, which was compiled from source. The server has 192 GB of RAM and 2 X 2.7 GHz CPUs, each with 12 cores. but I'm running only one squid worker. At the moment, there are 17 squid clients and the server is receiving 20025 http requests per minute. CPU usage is 62%. Squid is consuming about 12 GB of memory at the moment.
>>
>> Traffic is being redirected to this TPROXY using WCCP. Each once in a while, traffic redirection stops for about 5-10 minutes. When checking the WCCP clients on my Cisco switch, I see there is no WCCP client!
>>
>> During this pause time, running "squidclient mgr:info" freezes endlessly and does not return any results. Checking /var/log/squid/cache.log when this pause happens does not show any strange logs.
>>
>> When this condition happened last time, I shutdown squid using "service squid shutdown" and the init script reported that squid shutdown went well but running "ps aux | grep squid" returned the following:
>>
>> root 6900 0.0 0.0 103244 852 pts/0 S+ 12:05 0:00 grep squid
>> root 13006 0.0 0.0 38964 392 ? Ss Aug23 0:00 squid -f /etc/squid/squid.conf
>> squid 13009 63.1 25.0 50332228 49622116 ? Dl Aug23 3383:07 (squid-1) -f /etc/squid/squid.conf
>>
>> As you see, the worker squid kid is in uninterruptible sleep state and it remained in this state for a while. Even when squid seems to be working properly, the worker process may be in uninterruptible sleep for a short time, is this normal?
>>
>> Becuase squid worker process stayed in the uninterruptible sleep mode for quite long, I tried to stop it using "squid -k shutdown", which failed with the following:
>> squid: No running copy
>>
>> I'm wondering why this command is claiming "No running copy" while both the parent and kid squid processes are running!
>
> Squid might be dumping core during that interval. Have you checked the cache.log and/or looked for coredump files in the coredump_dir?
>
> Guy
Received on Tue Aug 27 2013 - 15:51:44 MDT