Re: memory-mapped files in Squid

Hi

> > I'd be very interested in finding out more about this. Who is working on
> > squid-FS?

> That would be me... I've got code here, two people have seen it, it's rough
> as guts, I'm hoping to do something more with it in the next couple of weeks
> (and yes, I've been saying that since before xmas :(. I unfortunately got
> hammered by other work, but that's slowing now...

I think that you need to take up the "quick release" motto... release your
code here. There is obviously interest, but it's difficult to start right
from the beginning.... I believe that if you release your code someone will
do some of the hacking that you want to do...

> storing filenumbers instead of names, where the number corresponds
> directly to the location on disk of the first block of the file. Keep

In the next few months a group of us here are going to be doing various
modifications to Linux here to get a really high-end cache going.

Currently our todo list (first to last) is a bit like this:

Scalable kernel performance for Internet servers under
        realistic loads - Gaurav Banga

        Since our current cache systems spend a LOT of time in select (we
        aren't using poll YET... perhaps that will alleviate things) this
        is our first stop.

Given that these are likely to be transparent proxies I believe that
        some function other than select is going to be unhappy...
        We are going to have to profile a bit and see what's next in
        line.

copyfile... this will reduce the cost of 'sends' as far as I know. I have
        spent about 15 minutes investigating it, though.

Creating a squid-fs. This is unlikly to be performed at the squid level:
        we are likely to do it in kernel mode: it is simply going to support
        things like 'open' and 'close' in their standard form... Given that
        ext2 is something like 6000 lines of code, a much simpler filesystem
        at the kernel level is going to be a reasonable amount of code.
        Romfs (very, very, very simple) in 2.2 is 709 lines.

I am currently trying to find a paper I had that spoke about the read/write
blocks sizes of modern disks...

I believe that your 4k figure is based on fastfs.ps by McKusic? That paper
is not exactly new: the principles apply....

Ok... it's now about 50 minutes later and I have found the paper... it was
on a friend's desk ;)

The Paper is entitled 'Embedded Inodes and Explicit Grouping: Exploiting
Disk Bandwidth for Small files'. It's by "Gregory R. Ganger and M. Frans
Kaashoek, Laboratory for Computer Science, Massachusetts Institute of
Technology"

If you are a usenix member you can download it here:
http://www.usenix.org/publications/library/proceedings/ana97/technical.html

If you aren't:
http://www.ece.cmu.edu/~ganger/papers/cffs.html

Another interesting URL is:
http://www.ece.cmu.edu/~ganger/disksim/index.html

Oskar

---
"Haven't slept at all. I don't see why people insist on sleeping. You feel
so much better if you don't. And how can anyone want to lose a minute -
a single minute of being alive?"				-- Think Twice