COSS+*UFS rebuild failures in squid-2.6

There's been some rebuild failures showing up in squid-2.6 when its used
with COSS and another store type (eg AUFS).

The basic reasoning is this: each storedir (aufs, diskd, ufs, and COSS
which I'm to blame for) assumes that if it sees an fresher object
during rebuild the older storeentry object is of the same type as
itself.

So, if AUFS sees an object which is fresher than something in COSS,
it'll try to manally unlink the COSS object - by using AUFS semantics.
This doesn't go down too well.

(COSS does something similar.)

Now, its all quite a bit of a hack. The main reason for it being a hack
stems from the UFS filenumber allocation stuff. It does it manually because

(a) it doesn't want the underlying files to be unlinked during a rebuild;
    it might be subsequently referenced by another swaplog entry which
    is 'fresh' (and, indeed, a completely new object!)
(b) cutting down on the spam going into the new swap log

So! How to fix it before Squid-2.6 is released? I believe there needs to be
a way to tell the store to release the object but not release the underlying
physical object in case that needs recycling. In the cause of COSS it won't
matter - the rebuilds don't happen from a logfile. In the case of the UFS
filesystems it'll matter - those are re-used.

So I'm proposing:

* adding a new store release call in store.c (storeReleaseSomething()), which
  is like storeRelease but doesn't cause a swaplog entry or unlink;
  (perhaps storeReleaseRecycle() ?)
* adding a new call to the SwapDir API which each storedir implements:
  it'll just be a call to repl->Remove() for the UFS dirs and it'll be a
  call to storeCossRemove() for COSS.

I'll leave it for a couple of days to let discussion happen but I'd like to get
this implemented early next week so I can test it out.

Thanks!

Adrian
Received on Thu Jun 29 2006 - 10:10:10 MDT