Re: async disk io

Adrian Chadd wrote:

> you will only be appending to the mem_node chain, right?

right.

> So you have
> a list of clients, you track the smallest offset off all clients, and
> you consider that block of memory non-wired (ie clean pages that are
> able to be reclaimed.)

Yes. This is how it is done today.

> This means deciding which memory pages in the mem_node list per object
> that are dirty (needing swapout), clean (having valid info on disk) and
> wired (are in use by an open client) is easy. Which means you don't need
> a refcount on eah page.

True, but in case of for example COSS, you might want to keep the data
in memory some time after the last client has finished, and given the
StoreIO isolation layer I don't think keeping the object locked until
the filesystem layer has actually finished storing the data is a good
approach. There is also the issue with requests being aborted while data
are being sent/stored and such, and how we handle cache hits. For some
(future) large objects it might also be required to allow holes in
memory where data is only kept on disk until the slower clients catches
up (or aborts, which is usually the case there).

It has been some time since I looked at how the Store I/O layer works,
so maybe this isn't really a issue. I have some vauge memory of counters
being updated when data was stored, and in that case the above shouldn't
be any problem.

> Now, this doesn't work for non-linear things like http range requests
> its a close enough estimation for now. Later on for streaming objects
> it will not be a valid assumption but that is later. :)

We really should not put in more limitations on how range requests can
be handled. Range is becoming a quite commonly used standard feature of
HTTP, and I expect there will be more plugins than Acrobat Reader which
makes out-of-order range requests.

/Henrik
Received on Sun Apr 09 2000 - 11:43:15 MDT