apache style squid?

Has anyone given any thought to building an apache style squid?

i.e.
        mmap() big chunk of memory for cache index.
        fork() off a bunch of children.

children do:
        accept(), and then process request in a single threaded
                fashion, locking if they need to access the shared
                memory holding the cache index.
        loop.

One of the children does:
        sleep on recvfrom(), and handle ICP requests.

At startup, you can also fork off a child to read in the cache index.

Advantages:
        No *&^&*^*&^&* blocking on disk! If one of the children does a
                slow operation, then it leaves all the other request
                processes running full speed.
        More efficent disk use. You can now queue multiple requests
                into the disk subsystem (because you don't need to
                wait until one completes). This means that if you have
                RAID'ed disks, or multiple disks, you actually get
                higher thruput out of them.
        MUCH simpler code. Don't need all the zillions of callbacks
                etc. You can write 'slow' code without need to split
                it into seperate functions.
        Smaller code. see above.
        Don't need large amounts of file descriptors per process. Each
                process is only handling 1 request at a time, so if
                it's using more than 15 file descriptors I'd be
                suprised.
        Much less polling. The ICP listener is in a seperate process,
                and can sleep on recvfrom() waiting for a packet.
                Ditto children listening for replies.
        On a 'slow' start where you need to stat each file, you can
                fire off one process per disk. MUCH faster startups.
        On a fast start, you can have a process dedicated to reading
                the log. again, much faster startup because it can
                occur at something close to disk speed, without
                significantly slowing down other request processing.
        Some of the above could be summerised by "the kernel scheduler
                automagically handles request load balancing for you".
        Some memory management wins in being able to alloc() a bunch
                of private memory, and then exit() the process, having
                the parent re-start you. I.e. guarenteed return of
                memory to the system.
        Don't need as much buffering when you sleep on read()'s,
                write()'s et al. i.e. lighter malloc()
                load. i.e. lower memory fragmentation in theory. :)
        Request processing doesn't stop dead when the logs need to be
                rotated. (a 3 million line log takes a while to dump
                to disk!)
        Possibility of instant startup. i.e. mmap() the index, and
                start using it immediately.

Disadvantages:
        Slightly more complicated memory management. Need to make sure
                shared memory is allocated out of the mmap() pool.
        Locking. In theory, this should be minimal. Only operations
                that change the index should need locking. There's
                also a little issue here with passing control if the
                lock you want is locked by someone else. Linux has
                sched_yield(), and I guess
                select(0, NULL, NULL, NULL, {0, 11000} ) would work
                for most other things.
        Lots more processes. Expect a couple of hundred processes for
                a busy cache I'd guess..
        Almost certainly need to allocate the mmap() for a fixed size
                (at least,if you want it portable)
        Slightly stiffer porting requirements. (now requires a working
                mmap() implementation, and a decent VM
                system. i.e. real copy-on-write)
        Possibly use more memory? This is a maybe. squid NOVM would be
                much more viable now that you can run the disk
                accesses concurrently.
        Uses a little more global sockets/file descriptors (each
                process has a log handle open, each has an ICP request
                socket open etc).

I had a quick start at hacking the code to do it, and was struck by
how much code I was deleting ("remove these 4 functions, replace
with 1 function that's a fraction of the size.. ").

Comments? Anyone else interested in coding this up?

Michael.
Received on Tue Jul 29 2003 - 13:15:43 MDT