Re: Hot object cache

On Mon, 2002-10-21 at 09:03, Henrik Nordström wrote:
> I for one do not like the current logics.
>
> Another proposed approach:
>
> I have proposed several times to only allow one client at a time until the
> object has been fully stored.. simplifies things greatly and avoids nasty
> races on bigish objects or stalled requests..
 
Whilst this will reduce the memory races, it won't benefit the hot
object cache directly. The change I am proposing (IMO) gives the hot
object cache for free.
 
> And for the hot object cache: This should really be managed as kind of a
> cache in it's own, not a artefact of the data stream as it is today.

To fix ranges, I have had to separate this out already. It's 90% done in
fix_ranges. It is missing the replacement logic, but each fragment in
the hot object store is able to be freed without affecting the fragments
before or after in the data stream. In other words, I agree with you.

> Fragments may be copied (by reference when memory management works) to the
> hot-object-cache where deemed useful. For a start I would propose this to
> be done on cache hits only, and only if the object size is below a given
> threshold. Some thought is needed in when to lock the hot object data
> considering objects who do not have a on-disk object or if the on-disk
> object gets deleted while there is a active client, but until index
> management is separated this is not likely to be a major concern..

Mm. I'm inclined to organically add this on to our current capability.
I'd do this by:
Freeing cache memory by memory LRU not memory object.
Recording LRU data in each datastream fragment.

I don't agree with the single-client change. To me, we will gain more by
separating the swap logic and reusing the multi-client logic, than
removing the multi-client logic and then having to reinstate it for
range combining in the future.

Let me put it this way: I need to alter the swapout code for fix ranges.
I have a couple of choices. Do you object to the one I suggested? We
can, in the fullness of time, do both yours and my suggestions - they
seem complementary to me.

Rob