On Wed, May 25, 2011 at 8:01 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> high and low water for the disk were at 85 and 95%, bumped it just to
>
> The watermark difference and total size determine how much disk gets erased
> when it overflows. Could be lots or not much. Very likely this is it. When
> the cache_dir reached high watermark it consumes a lot more disk IO and CPU
> erasing the cache until low watermark is reached.
Kind of what I thought but iowait didn't show enough to cause cpu
backup, so I was leary. CPU cycles sure, but the squid process shows:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30766 squid 20 0 6284m 6.1g 3068 S 13.9 38.8 91:51.50 squid
However most of today the resident memory sat at 6.4gb, as did all my
squid servers. So figured this is kind of the high end for memory
during busy times. Now it's dropped to 6.1gb, so I'm guessing we are
slowing down and thus squid is slowly shrinking its in memory
footprint.
> Look at using COSS either way for high number of small files.
Yes, I tried this and it was a complete fail. Load went through the
roof. Was a complete failure. this server should be complete overkill
for squid, 12 cores, 16gb mem, 12 15K sas drives. Even my boxes that
are using all 8 15K sas drives, have this weird load thang, (and let's
be honest, 4-5 load is nothing, but it's the weird spike and not
knowing what is causing the load and if it's something I can fix, is
the real issue).
Have tried various raid levels, including 0. Using LVM across all 8
disks, to match squid 4K write/read block size so that the VG/LVM
would write across all 8 disks equally etc.
earlier Coss config attempts
#### coss cache file system ####
#cache_dir coss /cache1/coss 30000 max-size=256000 block-size=8192 <--initial
#cache_dir coss /cache1/coss 3000 max-size=256000 block-size=2048 <--2nd testl
#cache_dir coss /cache2/coss 3000 max-size=256000 block-size=2048 <-2nd
#cache_dir coss /cache3/coss 3000 max-size=256000 block-size=2048 <-2nd
#cache_dir coss /cache4/coss 3000 max-size=256000 block-size=2048 <-2nd
#cache_swap_log /cache/%s
#### coss cache file system ####
> Could also be dumping cache_mem content to the disk for some reason. Though
> why it would do that without traffic is a mystery.
Misunderstood, or mistyped. This was during a peak period, but seems
to happen without significant increase in traffic. Like load is 4-5
now, and we are beyond peak.
├─squid(30763)───squid(30766)─┬─unlinkd(30767)
│ ├─{squid}(30768)
│ ├─{squid}(30769)
│ ├─{squid}(30770)
│ ├─{squid}(30771)
│ ├─{squid}(30772)
│ ├─{squid}(30773)
│ ├─{squid}(30774)
│ ├─{squid}(30775)
│ ├─{squid}(30776)
│ ├─{squid}(30777)
│ ├─{squid}(30778)
│ ├─{squid}(30779)
│ ├─{squid}(30780)
│ ├─{squid}(30781)
│ ├─{squid}(30782)
│ ├─{squid}(30783)
│ ├─{squid}(30784)
│ ├─{squid}(30785)
│ ├─{squid}(30786)
│ ├─{squid}(30787)
│ ├─{squid}(30788)
│ ├─{squid}(30789)
│ ├─{squid}(30790)
│ └─{squid}(30791)
>> see but made really no difference. My cache dirs are just 5gb each
>> (10gb total), on 150gb disks ya, but I don't really want to have to
>> weed through all of these files looking for an image.
>
> You wade through them manually?
No, :) just didn't want the system even with hash to have to deal
with a ton of files that are mostly stale within 10-30 minutes.
> Squid uses a hash, so its lookup time is good. You can tune L1/L2 for faster
> disk IO speeds with large caches.
l1/l2 cache? Have not considered or looked into it. New concept for me :)
>
> strace info during a spike should be useful to show what is going on.
>
installed, reading to see how and what information it will give me
> Amos
Thank you sir!
Tory
Received on Thu May 26 2011 - 03:27:11 MDT
This archive was generated by hypermail 2.2.0 : Thu May 26 2011 - 12:00:03 MDT