Re: Squid performance wish-list

Stewart Forster wrote:
>> Each file starts off with a 4K fragment that also contains that file's inode.
>> The inode simply lists the length of the file and 63 direct block pointers
>> and 64 single indirect block pointers for a inode length of 512 bytes (8 byte
>> pointers).

>Of course that should have been 4 byte pointers. This would allow us to
>index a disk that is 4K * 4G = 16TB in size (ie. Very big disks/stripesets)

This will do, for a while, I guess.

>3584 + 63 * 8192 + 64 * (8192 / 4) * 8192 = 1.0005 GB (big enough for any
>conceivable caching purposes for the next 10 years I'm guessing).

This I find dangerously low. Think everyone having 34Mb/s connections
and everyone pulling in MPEG movies.

On another note, what is to be gained by using direct and indirect
block pointers?
How about a much simpler scheme, a linked list like:

Say, 1KB block sizes, chunks consist of one or more blocks,
whereas every chunk starts with:

[ 1-byte: misc flags (reserved) ]
[ 3-bytes: length of this chunk in bytes ]
[ 4-bytes: blocknumber of the next chunk in the chain ]
[ 1016-bytes: data for this chunk ]
[ 1024-bytes: optional data to extend this chunk (no header) ]

This will allow partition sizes of: 4 TB
File sizes up to: 4 TB
Chunk sizes up to: 16 MB

The free space should probably consist of a linked list of these
chunks as well, whereas the allocation logic should round-robin across
the disk through the free-chain (this to allow max. defragmentation).

We might even go for 512 byte blocks,
probably gaining back some slack-disk space, thereby merely halving the
max filesystem size to 2 TB (which probably is still ok).
Heck, this could be fully configurable, since we don't have to be
compatible with anything, except ourselves, of course.

I'd say this is simpler, more space efficient, more extensible, and
faster than the FS you're proposing?

-- 
Sincerely,                                                          srb@cuci.nl
           Stephen R. van den Berg (AKA BuGless).
This signature third word omitted, yet is comprehensible.