Re: [PATCH] Solaris /dev/poll support for Squid 3 (how can I contribute)

  On 10/19/10 18:40, Alex Rousskov wrote:
> On 10/13/2010 04:21 AM, Peter Payne wrote:
>
>> BENCHMARKS
>>
>> Firstly, I wanted to address the benchmark questions as it made me
>> curious as to whether there really was an advantage in using /dev/poll.
>> I used the Apache Bench tool (that comes with HTTPD) to do my
>> benchmarks.
>>
>> I compiled a 32-bit Solaris 5.10 x86 of Squid version 3.1.8, and another
>> Squid version 3.1.8 with my patches. I shall call them "unpatched Squid"
>> and "patched Squid" respectively. Note that "unpatched Squid" was using
>> ordinary poll() and "patched Squid" was using /dev/poll.
>>
>> Where I used Apache bench with 1 simultaneous connection or even 8,000
>> simultaneous connections there was little difference between "unpatched
>> Squid" and "patched Squid". When all connections are busy there is no
>> performance gain.
>>
>> However the test that proved the most striking difference was
>> pre-establishing 8,000 TCP socket connections to Squid using a basic
>> Perl script. Then running Apache Bench with 1 connection making 100,000
>> keep-alive'd requests.
>>
>> unpatched Squid (using poll):
>> 2min23sec CPU user-time
>> 171sec to process 100000 Apache Bench queries
>>
>> patched Squid (using /dev/poll):
>> 0min22sec CPU user-time
>> 25sec to process 100000 Apache Bench queries
>
>
>
>> Clearly /dev/poll has a significant performance gain where there are
>> many idle TCP socket connections.
>
> On 10/13/2010 04:25 PM, Amos Jeffries wrote:
> > Oooh... Speed.
>
>
> Not really, at least not if you define "performance" or "speed" as
> "ability to sustain an X requests/second load".
>
> The above test is a classic best-effort test that, by design, cannot
> measure proxy's ability to handle load because its offered request
> rate depends on measured response time. During such a test, Squid
> drives the benchmark rather than the benchmark driving Squid.
>
> There are many real-world examples where a faster product (for any
> practical definition of "faster") would show drastically worse results
> during such a test.
>
> What the test does tell you is that an idle Squid became faster at
> processing a single transaction after the patch. "Significant
> performance gain" and similar conclusions are premature after a
> best-effort test like this one.
>
> Please do _not_ interpret my comments as negative towards the patch
> itself. I just want to minimize the chance that others will start
> running similar tests and jumping to the wrong conclusions regarding
> their own optimization ideas.
>
> Thank you,
>
> Alex.
>
Alex,

I completely agree. During testing there was little or no performance
gain when processing n busy connections (all busy). The only performance
benefit arose out of servicing few busy connections when many were idle.

As for the realism or effectiveness in the real world it may be
negligible. However there are situations when a connection is idle: when
waiting for a proxy to fetch a page from somewhere else, when holding a
connection open using keep-alive.

At any rate, it was a desired feature on the Squid 3 roadmap and was
already provided for Squid 2, and given that epoll and kqueue support
exists, it is added for completeness (with a LOT more comments in the
source too, initially I found reading the older code harder going).

Thanks,
Peter
Received on Wed Oct 20 2010 - 07:09:51 MDT