Re: squid rewrite

Andres Kroonmaa wrote:
> =

> As your thoughts are quite similar to those I have thought of for
> quite some time, I would like you to take a look at these ideas and
> comment aggressively. As it appeares to be too big to include with a
> mail, here is a URL: http://cache.online.ee/cache/Devel/design.txt

After a quick sweep throught your text it looks like a good
startingpoint, but... ;-) I need to take more time to read your text
carefully, after getting some sleep... =

As I am a bit restrictive on the use of shared memory, I beleive that
the bulk data transfers should be kept inside the same process as much
as possible. That is that the same process that fetches a object also
stores it on disk. (and reverse)

By using the same reasoning, ICP (protocol) and the store manager should
be glued together. the ICP queries requires ultra-fast lookups to see if
a object is cached, and ICP is fully dependent on that the store manager
keeps a consistent picture of the cache. (the other parts can actually
be made to continue to service even while the store-manager restarts).

The client lookups is not as time sensitive as the ICP lookups.

My basic ideas include something like this:

1. Each module should be independently restartable for soft recovery on
errors (errors will occur, only a matter of when...). Other modules
might stop servicing new requests or use a fall back routine while the
other needed module restarts.

2. Keep most of the processing inside the same process. Most notably the
data transfers. It is probably a bad idea to have the read and write =

split on two processes with a lot of IPC between them...

3. Store management and active processing should be separated, and store
management a lot more generalized. (on disk store management).

4. It should be possible to run a abritary number of copies of the
processing (frontend) parts. This is to distribute the load on several
processors, and to lower the FD usage per process (FD usage in total is
not as big problem).

5. FTP should probably be more integrated into Squid. The current ftpget
design is ...

6. There needs to be a good design for handling abritary sized objects
(unlimited by other than configuration rules).

I think it is time for a Moo meeting on future design of Squid. We
obviously have a lot of ideas on how it should be. (althought I feel
that I have lost touch with the inner core of Squid at the moment...)

---
Henrik Nordstr=F6m