Hot object cache

I for one do not like the current logics.

Another proposed approach:

I have proposed several times to only allow one client at a time until the
object has been fully stored.. simplifies things greatly and avoids nasty
races on bigish objects or stalled requests..

And for the hot object cache: This should really be managed as kind of a
cache in it's own, not a artefact of the data stream as it is today.
Fragments may be copied (by reference when memory management works) to the
hot-object-cache where deemed useful. For a start I would propose this to
be done on cache hits only, and only if the object size is below a given
threshold. Some thought is needed in when to lock the hot object data
considering objects who do not have a on-disk object or if the on-disk
object gets deleted while there is a active client, but until index
management is separated this is not likely to be a major concern..

Regards
Henrik

On 21 Oct 2002, Robert Collins wrote:

> On a related note though: I'd like to propose a change to the memory
> caching logic:
>
> Currently:
> we have n store clients that read data, and a swapout logic routine that
> 'chases' the bottom most store client, and finally we free the swapped
> out fragments once swapped out if the object is not a hot object.
>
> Proposed:
> The swapout logic becomes a store client in it's own right.
> Each fragment is marked as discardable once swapped out (if not a hot
> object).
> If a fragment is needed to satisfy a store client, it is retrieved from
> disk to memory, and immediately marked as discardable.
> Discardable fragments are discarded when memory is needed, according to
> a policy (say lru to start with)
>
> The change will involve a number of steps, these can be planned later.
> The important aspects are:
> 1) Restarting the cache will not prevent objects from being 'hot
> objects' - that is objects retrieved from disk can be cached in memory.
> 2) Objects with multiple readers that have one very slow and one fast
> will not consume inordinate amounts of memory - the intermediate pages
> will be able to be swapped to disk and back (if appropriate).
> 3) It is a critical step towards range combining, where we need to merge
> upstream and disk based data, by allowing disk data to be brought into a
> normal MemObject structure.
> 4) The current special logic for swapping out objects can go away and be
> replaced by simply starting a swapout client off. This makes the swapout
> process more loosely coupled, which will be important if/when we start
> updating disk metadata / caching ranges.
>
> Any objects to this design change? If not I'll implement it in
> fix_ranges and bring it across to HEAD from there.
>
> BTW: The fix_ranges effort is coming along nicely, we are nearly at the
> point of supporting range requests without always returning the full
> object again.
>
> Rob
>
Received on Sun Oct 20 2002 - 17:03:27 MDT