Re: Benchmarking Squid

On Tue, 2008-04-29 at 14:17 +1200, Amos Jeffries wrote:
> I've been giving the benchmarking a little more though over the last few
> days.
>
> The proposal from way back was to build a library of benchmarking results
> IIRC. We have some responses from people, but I have not seen anything
> like organisation or presentation of it.
>
> We need to publish some basic benchmarking configurations. Adrian has
> spoken before of his specific configs to stress certain parts of squid.
> Things like that which we can say to people "do this and let us know the
> results"
>
> When we have those benchmarking instructions ready we are going to need
> some feedback on the results. What details are needed to make useful
> graphs/tables etc of the results?
> - which benchmarking config was used
> - squid version
> - OS + Kernel
> - CPU speed
> - RAM
> - HDD speed + type?
> - NIC speed
> anything else?
> and what metrics should things be measured in? req per sec per CPU-MHz by OS?

FWIW, I am making [slow] progress towards the "standardized" setup which
we can use to benchmark Squid and publish the results. I hope to start
publishing in May.

A part of that work is preparing a set of modern workloads that folks
can use to submit more numbers. In my experience it is difficult to get
reliable/meaningful numbers so I am not very optimistic about collecting
quality public data (unfortunately), but I think we should still try.

The metrics depend on the workload, but in general you need at least
rate/throughput, response time, hit ratio (if caching), and errors. No
single metric is meaningful. You can optimize one and "freeze" others at
reasonable levels and workload often do that.

HTH,

Alex.
Received on Tue Apr 29 2008 - 05:38:51 MDT