RE: Storing partial responses

> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measurement-factory.com]
> Sent: Wednesday, 13 December 2000 4:20 PM
> To: Robert Collins
> Cc: squid-dev@squid-cache.org
> Subject: RE: Storing partial responses
>
>
> On Wed, 13 Dec 2000, Robert Collins wrote:
>
> > Hmm. Well I am definately heading to a (2) approach, because
> > performance tuning will need both FS and meta level activities.
>
<snip>
>
> An orthogonal example would be the delegation of
> responsibilities between hard drive controllers (fast operations on
> semantic-less easy-to-handle blobs/blocks) and Unix file systems
> (relatively slow operation on files). One could argue that if disks
> knew more about files, they would perform "better".

So the question I have: is the existing squid FS code a low level 'tool'
like the mechanicisms of a hard disk, or a medium level interface like
the unix file systems? I suspect it is at the file system level.

>
> In other words, sometimes it is better to keep the lower
> levels of the design fast and simple, while making slow but smart
> decisions at the meta-level. You do loose efficiency because of the
> communication between the layers, but you gain in overall design
> flexibility, which often proves more important, even performance wise.

I agree. See for example the acl result caching I've done in
auth_rewrite. It wasn't obvious that that could be done until some
sensible layers had been created, (namely,
access<->authentication<->user directories).

In the squid case here is how I see it layering out (corrections
requested :])

meta level - rfc 2616 decisions/acl/performance choices for request
satisfaction/choice of different FS's based on the FS capabilities.

---
fs level - efficient and fast object storage. Often tuned for a specific
purpose - ie small objects/big objects. The tuning implies smart choices
about object storage locations.
---
o/s level - high speed physical disk access.
Finally I think that the meta level will get too bogged down in the
mechanics of partial response storage if it is not abstracted (as per
your (2) before).
Rob