Re: [squid-users] Ramdisks

On 26 Nov 2001, at 16:19, Henrik Nordstrom <hno@squid-cache.org> wrote:

> On Monday 26 November 2001 10.45, Andres Kroonmaa wrote:
>
> > Maybe to you guys. To me it seemed pretty radical when I met it first.
>
> WAFL is basically what you get if you take a traditional filesystem like FFS,
> adopt it to the requirements of log structured and then drop the log to
> optimize space efficiency of snapshots and overwritten files.

 Seems you imply that there are much better approaches. Gotta dig around.

> Except that it still has many of the negative sides as well:
>
> * Removal of data is still a relatively heavy operation requiring writes to
> delete (or truncate) a file.

 hmm. I must have understood something wrong then.
 Idea in netapp's WAFL is to use snapshots on everything, on every FS write
 (which includes also removals). Several inode updates are needed indeed,
 but WA allows them to be batched together into single write op, thus any
 mod to the FS is mostly 1-diskop action. no? I recall that any kind of inode
 block can be located absolutely freely anywhere on the FS. If WAFL needs
 to modify 5 inode blocks located randomly all over the place, it will
 finally issue single write op of 5-block size (provided there is such
 free continuous chunk of course), or even 1-op per hundreds of fileops.
 Optimisations for read are another issue, and its assumed that read of
 those blocks is already done anyway.
 Why have you concluded removal as heavy in wafl?

> * The structure is likely to degrade with time as the files moves around,
> requiring the use of a defragmenter to keep the "tree of blocks" nice..

 True when looked as a traditional unix-FS. For squid this would be modified
 to use FIFO-like cyclic store, always writing at tail of log, wiping old
 stuff. In fact, as has been discussed in FIFO-LRU approach, even traditional
 FS could rewrite read-only accessed files to reorganize FS layout "for free".

> * Free space will most certainly degrade and fragment over time unless there
> is a defragmenter cleaning things up.

 Not an issue for Squid in fifo-lru case I think.

> * Poor packing of small objects (not tiny, but smaller than a couple of
> blocks), wasting a significant amount of cache memory and I/O bandwidth on
> small obejcts relative to their size.

 Well thats a matter of picking block size. 4K in WAFL may be too large
 for squid or news, but I'd say its even too small for large files. To
 optimize for small objects we could add special type of inodes describing
 files in smaller blocks (512b). The main principles of WAFL would stay
 the same.

> * The structure cannot easily be optimized for the specific application of
> caching.

 hmm, why not? I'd agree that it is not a best FS for small objects <4K
 (simple COSS would be much better), but as a general "fit-all" squid-FS
 it could be pretty good, imo.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Online
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Tue Nov 27 2001 - 03:12:13 MST