Re: [squid-users] Squid process crash every day, why?

On 16/03/2013 12:24 a.m., Feusi Remo (feus) wrote:
> Hi, I am new to the mailing list and have the following issue:
>
> Our squid crash every night between 01:00 and 02:00 CET.

What is Squid doing at those times?

> Mar 15 01:50:33 srv-app-904 (squid-1): Bungled (null) line 8: icap_retry deny all
> Mar 15 01:50:35 srv-app-904 squid[3589]: Squid Parent: (squid-1) process 3592 exited with status 1
> Mar 15 01:50:38 srv-app-904 squid[3589]: Squid Parent: (squid-1) process 4119 started
> Mar 15 01:51:32 srv-app-904 (squid-1): Bungled (null) line 8: icap_retry deny all
> Mar 15 01:51:32 srv-app-904 squid[3589]: Squid Parent: (squid-1) process 4119 exited with status 1
> Mar 15 01:51:35 srv-app-904 squid[3589]: Squid Parent: (squid-1) process 4141 started
> Mar 15 02:13:43 srv-app-904 (squid-1): Bungled (null) line 8: icap_retry deny all
> Mar 15 02:13:43 srv-app-904 squid[3589]: Squid Parent: (squid-1) process 4141 exited with status 1
> Mar 15 02:13:46 srv-app-904 squid[3589]: Squid Parent: (squid-1) process 4193 started

This usually shows up due to a problem in the auto-generated parser.
It can usually be resolved by rebuilding Squid in a new build directory.
if your RPM package was a binary one please contact the package provider
about this.

NP: I also suggest you upgrade to 3.2.9 or 3.3.3 ASAP.

> And here the interesting part from cache.log (with minus in the memory usage????):

Nothing to worry about. It is a well-known 32-bit wrap problem with
mallinfo() on 64-bit systems. Squid will operate just fine apart from
the memory reporting details being incorrect like this.

> 2013/03/15 01:50:33 kid1| Closing HTTP port 160.85.104.14:8080
> 2013/03/15 01:50:33 kid1| storeDirWriteCleanLogs: Starting...
> 2013/03/15 01:50:33 kid1| 65536 entries written so far.
> 2013/03/15 01:50:33 kid1| 131072 entries written so far.
> 2013/03/15 01:50:33 kid1| 196608 entries written so far.
> 2013/03/15 01:50:33 kid1| 262144 entries written so far.
> 2013/03/15 01:50:33 kid1| 327680 entries written so far.
> 2013/03/15 01:50:33 kid1| 393216 entries written so far.
> 2013/03/15 01:50:33 kid1| 458752 entries written so far.
> 2013/03/15 01:50:33 kid1| 524288 entries written so far.
> 2013/03/15 01:50:33 kid1| 589824 entries written so far.
> 2013/03/15 01:50:33 kid1| 655360 entries written so far.
> 2013/03/15 01:50:33 kid1| 720896 entries written so far.
> 2013/03/15 01:50:33 kid1| 786432 entries written so far.
> 2013/03/15 01:50:33 kid1| Finished. Wrote 799896 entries.
> 2013/03/15 01:50:33 kid1| Took 0.16 seconds (5155895.89 entries/sec).
> FATAL: Bungled (null) line 8: icap_retry deny all
> Squid Cache (Version 3.2.8): Terminated abnormally.
> CPU Usage: 1200.378 seconds = 742.532 user + 457.845 sys
> Maximum Resident Size: 10759744 KB
> Page faults with physical i/o: 12
> Memory usage for squid via mallinfo():
> total space in arena: -1523816 KB
> Ordinary blocks: -1587068 KB 98685 blks
> Small blocks: 0 KB 1 blks
> Holding blocks: 40796 KB 10 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 63251 KB
> Total in use: -1546272 KB 101%
> Total free: 63251 KB -3%
> 2013/03/15 01:50:38 kid1| Starting Squid Cache version 3.2.8 for x86_64-unknown-linux-gnu...
> 2013/03/15 01:50:38 kid1| Process ID 4119
> 2013/03/15 01:50:38 kid1| Process Roles: worker
> 2013/03/15 01:50:38 kid1| With 65535 file descriptors available
> 2013/03/15 01:50:38 kid1| Initializing IP Cache...
> 2013/03/15 01:50:38 kid1| DNS Socket created at 0.0.0.0, FD 7
> 2013/03/15 01:50:38 kid1| Adding nameserver 160.85.192.100 from squid.conf
> 2013/03/15 01:50:38 kid1| Logfile: opening log daemon:/var/log/squid/access.log
> 2013/03/15 01:50:38 kid1| Logfile Daemon: opening log /var/log/squid/access.log
> 2013/03/15 01:50:38 kid1| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
> 2013/03/15 01:50:38 kid1| Store logging disabled
> 2013/03/15 01:50:38 kid1| Swap maxSize 25600000 + 2097152 KB, estimated 2130550 objects
> 2013/03/15 01:50:38 kid1| Target number of buckets: 106527
>
>
> The Cache settings are as followed:
> cache_effective_user squid
> cache_effective_group squid
> cache_mem 2048 MB
> cache_dir aufs /var/cache/squid 25000 64 256
> maximum_object_size_in_memory 512 KB
>
> System Settings:
> - CentOS release 6.4 (Final)
> - uname -a: Linux srv-app-904 2.6.32-358.2.1.el6.x86_64 #1 SMP Wed Mar 13 00:26:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
> - rpm package: squid-3.2.8-1.el6.x86_64
> - MemTotal: 8061980 kB
>
> Thanks!
>
> Remo

I suggest you upgrade to 3.2.9 or 3.3.3 ASAP, it could be a few things,
including the 0-day which was published last week.

Amos
Received on Mon Mar 18 2013 - 00:42:24 MDT