Re: squid-smp: synchronization issue & solutions

On 11/20/2009 10:59 PM, Robert Collins wrote:
> On Tue, 2009-11-17 at 08:45 -0700, Alex Rousskov wrote:
>>>> Q1. What are the major areas or units of asynchronous code execution?
>>>> Some of us may prefer large areas such as "http_port acceptor" or
>>>> "cache" or "server side". Others may root for AsyncJob as the largest
>>>> asynchronous unit of execution. These two approaches and their
>>>> implications differ a lot. There may be other designs worth considering.

> I'd like to let people start writing (and perf testing!) patches. To
> unblock people. I think the primary questions are:
> - do we permit multiple approaches inside the same code base. E.g.
> OpenMP in some bits, pthreads / windows threads elsewhere, and 'job
> queues' or some such abstraction elsewhere ?
> (I vote yes, but with caution: someone trying something we don't
> already do should keep it on a branch and really measure it well until
> its got plenty of buy in).

I vote for multiple approaches at lower levels of the architecture and
against multiple approaches at highest level of the architecture. My Q1
was only about the highest levels, BTW.

For example, I do not think it is a good idea to allow a combination of
OpenMP, ACE, and something else as a top-level design. Understanding,
supporting, and tuning such a mix would be a nightmare, IMO.

On the other hand, using threads within some disk storage schemes while
using processes for things like "cache" may make a lot of sense, and we
already have examples of some of that working.

This is why I believe that the decision of processes versus threads *at
the highest level* of the architecture is so important. Yes, we are,
can, and will use threads at lower levels. There is no argument there.
The question is whether we can also use threads to split Squid into
several instances of "major areas" like client side(s), cache(s), and
server side(s).

See Henrik's email on why it is difficult to use threads at highest
levels. I am not convinced yet, but I do see Henrik's point, and I
consider the dangers he cites critical for the right Q1 answer.

> - If we do *not* permit multiple approaches, then what approach do we
> want for parallelisation. E.g. a number of long lived threads that take
> on work, or many transient threads as particular bits of the code need
> threads. I favour the former (long lived 'worker' threads).

For highest-level models, I do not think that "one job per
thread/process", "one call per thread/process", or any other "one little
short-lived something per thread/process" is a good idea. I do believe
we have to parallelize "major areas", and I think we should support
multiple instances of some of those "areas" (e.g., multiple client
sides). Each "major area" would be long-lived process/thread, of course.

Again for higher-level models, I am also skeptical that it is a good
idea to just split Squid into N mostly non-cooperating nearly identical
instances. It may be the right first step, but I would like to offer
more than that in terms of overall performance and tunability.

I hope the above explains why I consider Q1 critical for the meant
"highest level" scope and why "we already use processes and threads" is
certainly true but irrelevant within that scope.

Thank you,

Alex.
Received on Tue Nov 24 2009 - 23:13:51 MST