Re: [squid-users] Ramdisks

On Friday 23 November 2001 10.30, Joe Cooper wrote:

> This is where I got confused. So the log is in memory, but in
> 'infinite' as far as the the interface is concerned...buffer pressure
> flushes it in stripes as needed, and popularity can put an object back
> into the head of the log.

The log is a "infinite" area. Does not matter if it is kept in memory or on
disk, the structure is the same.

As it is a log, it also behaves like a journal, giving the nice benefit that
if you loose parts of the head you only loose the latest data, structure and
older data is intact.

> Will read. Having only read journalling filesystem research (and
> assuming log ~= journal) I wasn't aware this type of FS was well
> researched (I know about the similar object store in INND, but haven't
> studied it in detail either).

There has been quite a bit of research, but there is a lot more to research.
For example, very little attention has been given how to manage meta data in
a log structure. Most of the research has been based on a traditional FS and
then convert it to use a log structure as storage.

> Any reason why there can't be a super-thin delegation thread, that
> passes requests on to 'real' Squid processes based on a hash, so that
> cache data sharing isn't required? That's what we're doing at the
> network layer for dual processor Squid boxes, and it works very well.
> We've just got an iptables rule that splits on the last octet
> (0.0.0.0/0.0.0.128, and 0.0.0.128/0.0.0.128). It's not quite an even
> distribution, but it removes the requirement of sharing data.

Actually one of the ideas I am playing with. Such a design is a very good
approximation of having the "cache fs" as a kernel FS, shared by all writers.

> True...but we don't have a good filesystem design working yet. ;-)

True.

> I think picky writes is a 1 hour hack for me (50 minutes to find the
> spot to make the change, and 10 minutes to change it and add a
> squid.conf option), while the FS stuff is beyond my abilities at this
> point. I'm very much looking forward to using COSS, at least in
> experimental environments in the short term, but it still has at least
> one crasher (and possibly a second, but I won't bug Adrian about it
> until he's tracked down the first one I spotted). So even for folks who
> have the knowledge and have wrapped their head around the code, a good
> object store for Squid is a ways off in the future.

It is. And my "cyclical FS" is further away as it requires the store index to
be separated from "Squid".

> I'll drop the idea of picky writes, if and when COSS can give us
> practically free writes. ;-) As it is now, write bandwidth is a
> primary cause of the upper bound on Squids performance.

Exactly.

Regards
Henrik
Received on Fri Nov 23 2001 - 02:43:51 MST