Re: [squid-users] Squid Scalability

Gavin McCullagh wrote:
> On Mon, 06 Apr 2009, Amos Jeffries wrote:
>
>> Ah, sorry I meant CPU load as reported by Squid in %:
>>
>> "It can be extracted from the "general runtime information" or "info"
>> cachemgr page. It's the value marked "CPU Usage" "
>
> I'll hold off until a peak time and check. If it's similar, "top"
> generally shows relatively low cpu usage for the squid process.
>
> The munin CPU usage graph also gives a guide (the blue spikes are SRG
> running late at night).
>
> http://deathcab.gcd.ie/munin/gcd.ie/watcher.gcd.ie-cpu.html
>
>> Okay, we've reached the edge of my storage-specific performance
>> knowledge. I hope someone knows a bit more and can educate us both on a
>> good fix :)
>
> Mine too. The operating system is on linux software RAID1 partitions so I

Ah, there we probably have the answer as to why there is so much iowait.

You may want to find out how to determin RAID iowait vs other iowait and
see what shows up.

Though I only had 2x 250GB disks on 2.6GHz box RAID1, Squid maxed out
and started seriously lagging requests under 3 users (at around ~5Mbps
wild guess).

I shifted to a *slower* 1.8GHz box with single OS-shared disk and it now
hits serves 15 users without sweating and runs dozens of reverse-proxy
domains as a side job.

My review of RAID + Squid was overruled by some RAID experts with more
experience. I'm still puzzled how they got the evidence for
"performance: quite good" on software RAID though, maybe dual-core
minimum, mine are both singles.

> can swap out the smaller disk, pop in a second 1TB disk, sync the OS
> partitions and use the remainder as a second large cache. I intend doing
> that in the next week or so which should hopefully allow me to up our hit
> rate a bit.
>
> I'm not certain yet if the analysis I posted before is correct but if it
> is, it would appear a single SATA disk of this sort can only manage about
> 12-13% with our throughput (150-200GB per day, peaking around 40Mb/sec). I
> suspect if I came up with a way to improve our hit rate further (eg youtube
> caching), the iowait trouble would just get worse. Hopefully a second disk
> will allow this to expand further. After that, I guess I'll have to look
> at a new server.
>
> Gavin
>

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13
   Current Beta Squid 3.1.0.6