Re: apache style squid?

--MimeMultipartBoundary
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Tue, 7 Oct 1997, Andres Kroonmaa wrote:

> Both models introduce locking and IPC troubles.

Note that there is a reason for locking around accept() in any server
where multiple threads/processes are going to use it. Certainly if you're
accept()ing on multiple sockets, this is a limitation in the posix
interface. I also try to explain why Apache locks around accept() even
when it's only a single socket, and the OS should handle it fine. See
<http://www.arctic.org/~dgaudet/apache/perf> and skip down to the part
about serialization.

What you really want in many of these situations is a pipe on which you
read completed i/o events... which is exactly what NT's completion ports
are all about. select() is inefficient if multiple threads/processes are
running it on the same descriptors... since it has a "wake all" semantic
-- everything is awakend when something happens. In fact in many kernels
things like accept(), which definately has a wake-one semantic from the
user point of view, is implemented as a wake-all in the kernel and most of
the sleepers go back to sleep.

BTW other than locking around accept(), apache doesn't need to do any IPC
that's expensive. It keeps a scoreboard indicating what state each child
is in, and the parent watches the scoreboard occasionally to figure out if
there's enough or too many children. The children even use the scoreboard
to let the parent know about timeouts, so that in the most common code
path (i.e. no timeouts happen) the children don't have to make any system
calls just to set timers.

> It may be, yes. But to me, on Solaris it is really right to go threads.

It definately is.

> Very nice feature for client reads when hits are serviced is to
> mmap diskfile to memory and then simply issue full write to client from
> this memory in one go of object size. No kernel-userspace buffer copies.
> Kernel does file io and socket writes.

Use writev() to combine this with writing the headers. But you'll run
into buffering and timeout issues I think. I thought squid did this
already?

> > Yes and no. You do get advantages from true private resources. I'll
> > grant that threads do avoid some of the complexities, but do it by
> > trading off on the safety issues.
>
> What do you mean by safety tradeoffs?

One redeeming "feature" of Apache's model is that it is very segfault
tolerant. For something like squid this is not as much of an issue
because you have a very tight well controlled code base. For apache with
all the user modules, which frequently are, uh, lame, segfault protection
is something of an issue. We try to keep the core code segfault free of
course, but I've seen sites with custom modules that cause segfaults.
Usually at a low enough rate to not be worth fixing (from their p.o.v.).

-- 
Dean Gaudet, Performance Analyst, Transmeta Corp.
--MimeMultipartBoundary--