Re: [squid-users] Squid performance profiling

On 06/20/2013 10:47 PM, Ahmed Talha Khan wrote:
> On Fri, Jun 21, 2013 at 6:17 AM, Alex Rousskov wrote:
>> On 06/20/2013 02:00 AM, Ahmed Talha Khan wrote:
>>> My test methodology looks like this
>>>
>>> generator(apache benchmark)<------->squid<------>server(lighttpd)
>> ...
>>> These results show that squid is NOT CPU bound at this point. Neither
>>> is it Network IO bound because i can get much more throughput when I
>>> only run the generator with the server. In this case squid should be
>>> able to do more. Where is the bottleneck coming from?

>> The "bottleneck" may be coming from your test methodology -- you are
>> allowing Squid to slow down the benchmark instead of benchmark driving
>> the Squid box to its limits. You appear to be using what we call a "best
>> effort" test, where the request rate is determined by Squid response
>> time. In most real-world environments concerned with performance, the
>> request rate does not decrease just because a proxy wants to slow down a
>> little.

> Then the question becomes why squid is slowing down?

I think there are 2.5 primary reasons for that:

1) Higher concurrency level ("c" in your tables) means more
waiting/queuing time for each transaction: When [a part of] one
transaction has to wait for [a part of] another before being served,
transaction response time goes up. For example, the more network sockets
are "ready" at the same time, the higher the response time is going to
be for the transaction which socket happens to be the last one ready
during that specific I/O loop iteration.

2a) Squid sometimes uses hard-coded limits for various internal caches
and tables. With higher concurrency level, Squid starts hitting those
limits and operating less efficiently (e.g., not keeping a connection
persistent because the persistent connection table is full -- I do not
remember whether this actually happens, so this is just an example of
what could happen to illustrate 2a).

2b) Poor concurrency scale. Some Squid code becomes slower with more
concurrent transactions flying around because that code has to iterate
more structures while dealing with more collisions and such.

There is nothing we can do about #1, but we can improve #2a and #2b
(they are kind of related).

> best effort tests also
> give a good measure of what the proxy(server) can do without breaking
> it.

Yes, but, in my experience, the vast majority of best-effort results are
misinterpreted: It is very difficult to use a best-effort test
correctly, and it is very easy to come to the wrong conclusions by
looking at its results. YMMV.

BTW, a "persistent load" test does not have to break the proxy. You only
need to break the proxy if you want to find where its breaking point
(and, hence, the bottleneck) is with respect to load (or other traffic
parameters).

> Do you see any reason from the perf results/benchmarks
> why squid would not be utilizing all CPU and giving out more requests
> per second?

In our tests, Squid does utilize virtually all CPU cycles (if we push it
hard enough). It is just a matter of creating enough/appropriate load.

However, if you are asking whether Squid can be changed to run faster
than it does today, then the answer is yes, of course. There is still a
lot of inefficient, slow code in Squid.

HTH,

Alex.
Received on Fri Jun 21 2013 - 05:41:30 MDT