Re: copy-on-write

andre@ml.ee writes:

>> Hi
>>
>> Looking at
>> http://squid.nlanr.net/Squid/Devel/todo.html
>> http://squid.nlanr.net/Squid/Devel/Todo/9703122.txt

STUB CHILD PROCESS

Once the main Squid process grows in size, it should never fork().
Squid needs a stub child process from which other children can
be spawned. Features:

    1) the interface should return a pair of FDs, one for reading
       and one for writing. Then we might be able to make the
       IPC method more configurable, eg use Unix domain sockets if
       supported.
    2) The stub process should open 'cache.log' on stderr. Alternatively
       the individual child processes (e.g. ftpget) might open the
       cache.log based on an environment variable. When ever Squid
       closes and reopens the log, the child processes would have to
       be signalled to do the same.
    3) The stub process must not exec an non-Squid executable.
       It should be limited to ftpget, dnsserver, and redirector.

> "Copy-On-Write" relies on page-protection "read-only". After a fork()
> you are guaranteed to have exact copy of all proccess image, including
> read/write data parts. Now, to implement "Copy-On-Write" for all data
> squid uses, OS should mark all parent proccess pages to be read-only
> just for the sake of child process - this is quite an overhead and as
> parent process still probably modifies most pages all the time, the
> copy-on-write would still duplicate all pages, but later, via pagefaults.
>
> As all this would slow down both processes in overall, OS-es do not do
> that for data pages, instead they copy full proccess images. To overcome
> this overhead, algoritmic changes are needed, and this is exactly what
> stub process is for.

I started to code this (the stub process thing) yesterday but
got stuck a bit. The best option I could come up with was:

    Use TCP sockets. All Squid IPC would be over TCP sockets. Some
    people are uncomfortable with this. One problem is that others on
    the same host might still make connections to the stub process. I
    could bind Squid to some port before forking the the stub and have
    the stub dump connections from other ports
 
Unix domain sockets (non blocking only?) are broken on some OS'es.

FD passing may be another option, but I don't know how portable it is.

Duane W.
Received on Tue Jul 29 2003 - 13:15:41 MDT