Re: store module abstractions

----- Original Message -----
From: "Kevin Littlejohn" <darius@bofh.net.au>
To: "Robert Collins" <robert.collins@itdomain.com.au>
Cc: <squid-dev@squid-cache.org>
Sent: Sunday, February 18, 2001 6:01 PM
Subject: Re: store module abstractions

>
> >>> "Robert Collins" wrote
> > I need to get my hands dirty on this to answer. You've probably got a better
> idea about it than me.
> >
> > I had vague thoughts along the line of
> > storefs - physical layout of the store & rebuild logic & metadata logic
> > storeio - o/s level calls to read/write/create/delete.
>
> This _does_ run straight into the work Adrian's doing on modio, so I might
> withhold from commenting _too_ much until he chimes in ;)

Ok.

> However, I'll foreshadow some problems with any of these splits - fs'es like
> reiser and sfs throw spanners in the works be trying to implement everything
> themselves or trying to reach into the guts of how squid maintains it's own
> list of available files.

I don't see that as a problem. I'm not suggesting that all storefs level layers use the third layer. As an analogy, if you choose to
you can write a storefs module now that ignore the replacement policy module and implements its own. This might make sense with COSS
for instance if FIFO is all you can efficiently support.

> My preference at the moment is a very thin layer implementing store* calls -
> storeDirRebuild, storeRead, storeWrite, storeUnlink, etc. - and then some hard
> thought about a good interface for it to talk to reiser/sfs/coss/ufs. It'd be
> nice to raise async up out of the bottom layer, so fs'es can be given
> async-ness in a braindead manner, but I suspect that only applies to ufs -
> sfs implements async-ness itself, for instance.

Right. That sounds like what i'm talking about. That layer is async itself.
It *should not* intergace to reiser/coss IMO. They don't have process's analagous to dirRebuild do they?

> modio is supposed to be providing us with a standard way of feeding the data
> from the fs back up into squid - that will help immensely in defining what
> layers should exist, I'd envisage the storeRead/storeWrite interface would
> manage a lot of that. However, that may lead us toward excessive memcpy'ing
> to bring the data back up (or down, for writing).

Right. Well henrik's recent suggestion for refcounts in the io path should go aways to reducing memcpying, although for things like
diskd that may be unavoidable (like I know diskd - ha! ).

> If you're not planning on diving in immediately, then I'll keep prodding at
> sfs with a view to understanding how it works currently, and keep hassling
> Adrian to gimme more info about the data path for modio wrt fs ;)
>
> KevinL
>

No chance of me digging in. I'd be juggling too much. Please keep me posted on progress though :]

Rob
Received on Sun Feb 18 2001 - 01:11:42 MST