-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Sat, 11 Aug 2007 17:24:10 -0300 (BRT)
"Michel Santos" <michel@lucenet.com.br> wrote:
>
> Tek Bahadur Limbu disse na ultima mensagem:
> > Michel Santos wrote:
> >> Tek Bahadur Limbu disse na ultima mensagem:
> >>> diskd indeed seems to fail under load especially when approaching
> >>> 200/300 requests per second.
> >>
> >> are you sure this numbers are correct? where do you get them from?
> >
> > Hi Michel,
> >
> > I am getting these numbers from one of my busy proxy server. At peak
> > times, I get anywhere from 150-200 requests per second. However to cross
> > the 300 mark, it only happens when 1 or 2 of my other proxy servers go
> > down and then our load balancer redirects web requests to whichever
> > proxy server is up and functioning.
> >
>
> so you get 12000/min right? But when I asked where you get them from I
> wanted to know how you count them, snmp? cachemgr?
Hi Michel,
I get those statistics by snmp for creating the MRTG and RRD graphs. Yes at peak times, I get 12000 requests/min. Also if I need to see them in real time, I get it with squidclient:
#squidclient mgr:5min | grep client
client_http.requests = 205.725081/sec
client_http.hits = 19.516516/sec
client_http.errors = 0.000000/sec
client_http.kbytes_in = 168.768699/sec
client_http.kbytes_out = 1473.785309/sec
client_http.all_median_svc_time = 1.461314 seconds
client_http.miss_median_svc_time = 1.542425 seconds
client_http.nm_median_svc_time = 0.001789 seconds
client_http.nh_median_svc_time = 1.542425 seconds
client_http.hit_median_svc_time = 0.023168 seconds
>
> how much mem the server has installed?
Most of them have 1 GB memory
>
>
> >
> > I guess that I may have to really commit my time and resources to find
> > out if other factors could be causing this to happen.
> >
> > Haven't you faced any automatic restart of your Squid process. Does that
> > mean that your Squid process uptime is months?
> >
>
> never dies by it's own, my problem are power problems and loose human
> endpoints (fingers) :)
>
> what is you kern.maxdsiz value?
It's the default value of 512 MB. I guess I may have to increase it to say 768 MB.
I can put the following value in /boot/loader.conf:
kern.maxdsiz=754974720
>
> How much memory squid is using just before it crashs? is it using swap?
> what ipcs tells you then or under load?
Squid could be using somewhere between 500 to 700 MB of memory before it crashes.
It was not using swap.
Currently, ipcs tells me:
#ipcs
Message Queues:
T ID KEY MODE OWNER GROUP
q 720896 61288448 --rwa------ squid squid
q 720897 61288449 --rwa------ squid squid
q 393218 61288452 --rwa------ squid squid
q 393219 61288453 --rwa------ squid squid
Shared Memory:
T ID KEY MODE OWNER GROUP
m 720896 61288450 --rw------- squid squid
m 393217 61288454 --rw------- squid squid
Semaphores:
T ID KEY MODE OWNER GROUP
>
> >
> > They have been in production for years and each of their average uptime
> > is about 120 days. As far as the load is concerned, my CPU usage never
> > goes above 30-40% but sometimes my memory usage crosses 80% of it's
> > capacity though.
> >
>
> what hardware is it? Which freebsd version your run? And how is your
> layout, standalone proxy server, gateways or cache hierarchy?
Most of them are Dell SC-420 machines:
CPU 2.80GHz (2793.09-MHz K8-class CPU)
Hyperthreading: 2 logical CPUs
OS: FreeBSD-6.0-6.1 (amd64).
I have a dozen machines. They are somewhat in a farm where they are acting as siblings to each other on the same network. In front of them is an Alteon load balancer.
>
>
> > By the way, do you have some optimal settings which can be applied to
> > diskd? Below are some values I use:
> >
> > options SHMSEG=128
> > options SHMMNI=256
> > options SHMMAX=50331648 # max shared memory segment size (bytes)
> > options SHMALL=16384 # max amount of shared memory (pages)
> > options MSGMNB=16384 # max # of bytes in a queue
> > options MSGMNI=48 # number of message queue identifiers
> > options MSGSEG=768 # number of message segments
> > options MSGSSZ=64 # size of a message segment
> > options MSGTQL=4096 # max messages in system
> >
> > Correct me where necessary.
> >
>
>
> that does not say so much, better you send what comes from sysctl kern.ipc
#sysctl kern.ipc
kern.ipc.maxsockbuf: 262144
kern.ipc.sockbuf_waste_factor: 8
kern.ipc.somaxconn: 8192
kern.ipc.max_linkhdr: 16
kern.ipc.max_protohdr: 40
kern.ipc.max_hdr: 56
kern.ipc.max_datalen: 120
kern.ipc.nmbclusters: 0
kern.ipc.piperesizeallowed: 1
kern.ipc.piperesizefail: 0
kern.ipc.pipeallocfail: 0
kern.ipc.pipefragretry: 0
kern.ipc.pipekva: 98304
kern.ipc.pipes: 12
kern.ipc.maxpipekva: 17170432
kern.ipc.msgseg: 512
kern.ipc.msgssz: 64
kern.ipc.msgtql: 4096
kern.ipc.msgmnb: 16384
kern.ipc.msgmni: 40
kern.ipc.msgmax: 32768
kern.ipc.semaem: 16384
kern.ipc.semvmx: 32767
kern.ipc.semusz: 104
kern.ipc.semume: 10
kern.ipc.semopm: 100
kern.ipc.semmsl: 60
kern.ipc.semmnu: 30
kern.ipc.semmns: 60
kern.ipc.semmni: 10
kern.ipc.semmap: 30
kern.ipc.shm_allow_removed: 0
kern.ipc.shm_use_phys: 0
kern.ipc.shmall: 16384
kern.ipc.shmseg: 128
kern.ipc.shmmni: 32
kern.ipc.shmmin: 1
kern.ipc.shmmax: 16777216
kern.ipc.numopensockets: 3502
kern.ipc.maxsockets: 32768
kern.ipc.nsfbufsused: 0
kern.ipc.nsfbufspeak: 0
kern.ipc.nsfbufs: 0
>
> anyway you probably should not limit SHMMAX but set SHMMAXPGS so then
> SHMMAX is correctly calculated and no need to compile them, that are
> sysctl tunables
>
You mean set SHMMAXPGS using sysctl or compile it? Also what the best value for SHMMAXPGS?
> I believe any wrong value would not make your server crash, worst case
> that your msg queues get stucked what would then put squid's disk r/w
> access on hold but not to crash, well, let say I never saw a server
> crashing for ipc congestion, the client simply stops communicating
I hope so!
Thanks once again for your tips and suggestions.
Thanking you..
>
> Michel
> ...
>
>
>
>
> ****************************************************
> Datacenter Matik http://datacenter.matik.com.br
> E-Mail e Data Hosting Service para Profissionais.
> ****************************************************
>
>
- --
With best regards and good wishes,
Yours sincerely,
Tek Bahadur Limbu
(TAG/TDG Group)
Jwl Systems Department
Worldlink Communications Pvt. Ltd.
Jawalakhel, Nepal
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (FreeBSD)
iD8DBQFGvrAEfpE0pz+xqQQRAmqSAKCsTR0js6bHNe4y+Eup2HHHNnItTACgyu4J
Avy+vw4Lq2yTB0YY0zW3riY=
=g3a2
-----END PGP SIGNATURE-----
Received on Sun Aug 12 2007 - 01:00:16 MDT
This archive was generated by hypermail pre-2.1.9 : Sat Sep 01 2007 - 12:00:03 MDT