RE: [squid-users] squid on redhat6 crash

Eliezer

Great thanks for the reply. From briefly dump information I only get is:
Crash comman: is squid -f /etc/squid/squid.conf
Reason: is Process /usr/sbin/squid was killed by signal 6 (SIGABRT)
Kernel: is 2.6.32-279.2.1.el6.x86_64 (Red Hat Enterprise Linux Server release 6.3 (Santiago))
Pwd: /var/cache/squid

Limits:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 256718 256718 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 256718 256718 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us

Below to answer your question:

>you mean load by req\s?memory?cpu?disk i\o?

The load I mean is when squid client increase; I found it only crash between 9am-9pm, did not crash at night time.

>how many requests per sec?

Now, I can not check, it is out of service now.

>squid 3.1 self compiled or from rpm?

It is from system rpm.

>using any special helpers?how many child process(for helpers)?

Squid_ldap_auth, 10 child process, do not anything else.

>"squid -v" output?
]# squid -v
Squid Cache: Version 3.1.10
configure options: '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-arp-acl' '--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL,DB,POP3,squid_radius_auth' '--enable-ntlm-auth-helpers=smb_lm,no_check,fakeauth' '--enable-digest-auth-helpers=password,ldap,eDirectory' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-referer-log' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl' '--enable-storeio=aufs,diskd,ufs' '--enable-useragent-log' '--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=squid' '--with-filedescriptors=16384' '--with-dl' '--with-openssl' '--with-pthreads' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' 'LDFLAGS=-pie' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' --with-squid=/builddir/build/BUILD/squid-3.1.10

>squid.conf ?
Related part:
auth_param basic program /usr/lib64/squid/squid_ldap_auth ........
auth_param basic children 10
auth_param basic realm xx
auth_param basic credentialsttl 15 minutes
authenticate_ip_ttl 5 minutes
acl all src all
icp_access deny all
acl_uses_indirect_client off
log_uses_indirect_client off
http_port xxx
hierarchy_stoplist cgi-bin ? squid-internal-
cache_mem 2048 MB
maximum_object_size_in_memory 1 MB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir aufs xxxx
maximum_object_size 64 MB
cache_store_log none
dns_retransmit_interval 1 second
dns_timeout 1 minute
ipcache_size 8192
ipcache_low 95
ipcache_high 98
memory_pools_limit none
forwarded_for off
uri_whitespace deny
coredump_dir /var/cache/squid
pipeline_prefetch on
sleep_after_fork 500000

>transparent proxy?
no

>"ulimit -Sa" output?
ore file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256718
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

>"ulimit -Ha" output?
# ulimit -Ha
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256718
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

many regards

Julie

-----Original Message-----
From: Eliezer Croitoru [mailto:eliezer_at_ngtech.co.il]
Sent: Tuesday, 21 August 2012 12:33 AM
To: squid-users_at_squid-cache.org
Subject: Re: [squid-users] squid on redhat6 crash

On 8/20/2012 4:01 PM, Julie Xu wrote:
> Hi
>
> Used redhat6' squid 3.1, and the system is crashing when there is load increased.
>
you mean load by req\s?memory?cpu?disk i\o?
how many requests per sec?

> Aug 20 14:30:01 proxy-server squid[2450]: Squid Parent: child process
> 6591 exited due to signal 6 with status 0 Aug 20 14:30:04 proxy-server
> squid[2450]: Squid Parent: child process 7091 started Aug 20 14:46:40
> proxy-server abrt[22338]: Saved core dump of pid 7091
> (/usr/sbin/squid) to /var/spool/abrt/ccpp-2012-08-20-14:46:36-7091
> (668131328 bytes)
>
> Briefly look the dump directory, only thing I found is the direct is the cache direct, there is a coredump file, try to use crash to read it, not successful.
>
> Could anyone advise me what is possible reason for the crash, and the solution?
squid 3.1 self compiled or from rpm?
using any special helpers?how many child process(for helpers)?
"squid -v" output?
squid.conf ?
transparent proxy?
"ulimit -Sa" output?
"ulimit -Ha" output?

the more data you will provide us we can pinpoint the reason for the problem.

Regards,
Eliezer
>
> Thanks in advance
>
> Julie
>

--
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations eliezer <at> ngtech.co.il