Re: memory-mapped files in Squid

> "squid-FS and alikes".
> In fact, squid should be rewritten somehow so that it has some
> generalised (object)file io api that could be more easily modified. Then
> it would be possible to experiment with many different algoritms.
> Currently, if you want to try your stuff out, you'd have to rewrite
> pretty much of squid, and then you are your own.

The interface it's currently got, while not the best, isn't absolutely
atrocious, tho - depending a lot on exactly _where_ you want to interface,
of course, and whether or not you want to support asyncio alongside a new
fs.

> I'd suggest looking into using just plain swapfiles in bunch, either
> ontop of UFS or on raw partitions. Implement block allocation and
> maintenance yourself, tightly coupled with squid' needs. There is
> one work in progress for squid-FS that looks very interesting, and
> if you like, I can present you my own ideas on the subject some time.
>
> I'd be very interested in finding out more about this. Who is working on
> squid-FS?

That would be me... I've got code here, two people have seen it, it's rough
as guts, I'm hoping to do something more with it in the next couple of weeks
(and yes, I've been saying that since before xmas :(. I unfortunately got
hammered by other work, but that's slowing now...

If you're curious about the design, search back in the mailing list
archives for discussions involving myself, Stew Forster, and others - a
fair whack of the design was laid out there. Basically, it's aimed at
storing filenumbers instead of names, where the number corresponds
directly to the location on disk of the first block of the file. Keep
the amount of metadata (length, next block pointers, etc) low, and you
can fill the first block with almost 4K of actual data - enough to satisfy
most (around 50%, anyway) requests in one disk read. Other issues (such
as fragmentation) are partially dealt with in previous discussions and
in doccos I've got here, and partially a case of 'suck it and see'...

When I'm happy the code isn't a total embarassment, I'll throw it out to
the wolves, and gratefully accept any comment you all might have - but
the state it's in right now, I'm not happy with. I'm aiming for within
the month to get it integrated properly and running without mass crashes
etc.

KevinL

--------------- qnevhf@obsu.arg.nh ---------------
Kevin Littlejohn,
Technical Architect, Connect.com.au
Don't anthropomorphise computers - they hate that.
Received on Tue Jul 29 2003 - 13:15:55 MDT