Re: client-store interface

On Thu, 2003-10-30 at 10:44, Henrik Nordstrom wrote:
> On Tue, 28 Oct 2003, Robert Collins wrote:
>
> > I think we would benefit from a extension to the store client interface,
> > to support notification of new data arriving (as opposed to the
> > completion of read requests). This would be used for:
> > 1) multipart responses in the future, where the order of offset-length
> > data isn't predictable.
>
> This I don't quite get how this would help.. at least not without
> additional concepts. If doing this then there needs to be a somewhat
> bigger scope considered. See below for some discussion on what I think
> about Range processing how/when/why.

Sure..

> > 2) passing through range responses where the range processing fail for
> > whatever reason.
>
> This I do not get at all..

> Range processing of incoming data (merging of partial responses etc) needs
> to be done in the HTTP server side. The client side should not need to
> care about this, and neither should the store (at least not initially).

Yes, and it is... but.

> Range reply generation should be done at the client side, asking the store
> for the data it needs. This requires a conditional open API call where the
> store client can ask if the requested ranges are available in the cache,
> and if we want to support non-linear ranges then the API needs to support
> some kind of random reads. But hold that thought about random reads a
> little (there is additional complications).

The API supports random reads now.

> If not all requested data is available in the cache then my opinion is
> that the request should be a handled as a cache miss and processed by the
> server side code.

Sure, even a cache miss suffers this issue though.

> In neither case should the store need to know the details of HTTP range
> processing. All the store needs to know is that a object consists of
> * metadata used by the store
> * a header portion
> * a body, possibly sparse
>
> The API for both reading and writing from the store needs to reflect this,
> and handle reading/writing of the body in arbitrary order.

Yep, again it does now.

> The servers side, when merging partial responses, need to be able to ask
> the store in detail on what we have in the cache allowing it to construct
> the proper ranged request to fill in the blanks where required.

This is in the todo - we don't have any merging logic in place at the
moment.

> Merging of partial responses should (at least until we have a middle
> layer, allowing forwarding between server/client without using the store)
> be done into a new store object. This is important to guarantee
> consistency and a clean path, and avoids a very sticky web of issues if
> there is multiple clients asking for the same object etc. The drawback is
> that we will loose caching of "concurrent" ranges from different requests
> (i.e. due to download accelerators etc).

I don't agree on this - but it's largely irrelevant at this point, as
I've no short term plans for merging of ranges - when someone starts to
work on this we can finalise the design.

> From this I propose the following store client API guidelines:
>
> * header and body data handled separately, not as a single stream.

Yes - that is 90% in place now. Object offsets are done from the object,
not the header start.

> * when a store client is initiated it must indicate if a ranged/partial
> response is acceptable, and in such case what ranges it wants.

This is sort of in place - the client simply asks for the ranges it
wants, one at a time. The range metadata is passed through to http.cc,
which then obtains those ranges or the full object as appropriate and as
per policy.

> * the store then gives the client the entity header

yes.

> * followed by ranges of body content. The ranges of body content should
> preferably be returned in the same order as asked for when the store
> client is initiated (SHOULD on cache hits, MAY on cache misses while using
> the store for forwarding of data)

The data ordering is only a SHOULD for multipart requests, it's a MUST
for single-part requests.

Before we get into your examples, let me say that the design I put into
squid-3.0 is almost exactly what you describe. The issues are all corner
cases.

Case 1:
the upstream returns a partial response with invalid Range metadata -
the range request (such as -50) cannot be normalised for whatever
reason. Robust handling dictates that we simply pass through the data
returned, even though we cannot make head nor tail of it. Alternatively
we could display an error page.

Case 2:
the upstream returns multi-part data and we parse it. The order doesn't
match what we requested (which is valid).
Our request for specific bytes in the store may end up deadlocking:
- if the object isn't cached.
- the data we need to send isn't inside the readahead gap.
- the memory flush algorithm is aware of non lowest-highest retrieval
patterns.
then we will never read the bytes the client needs.

Case 3:
the upstream response isn't actually compatible with what the client
requested - and ETag violation. Again, dealing with a buggy server, we
should pass through the data or display an error. This appears to be the
case that is in bug #624. (I've a branch
(robertc@squid-cache.org--squid/squid--fix-ranges--3.0) dedicated to
addressing this issue)

The notification approach (combined with a couple of query methods that
already exist) allows the client side to examine the memory store as
data arrives, rather than needing to know which data is going to arrive
next.

> Lets dive into a few examples here to better illustrate the requirements:
> [based on forwarding via the store design]
>
>
> ** Full object cached. Range request received. **
>
> A store client is initiated, asking for the required ranges.
>
> The store then returns the ranged body data in the order requested.
>
> If there then is a problem reading in some of the content from the store
> then this should be handled as any read error.

Yep, done.

> ** No object cached. Range request received **
>
> Cache miss. Request forwarded.
>
> Server-side stores the header component, and then the reply body ranges as
> received.
>
> Client-side gets the ranges as stored by the server-side, and transforms
> this back into a suitable HTTP reply.

Yep, Done.

> ** Partial object cached. Normal request **
>
> Partial cache miss. Request forwarded but with a reference to the partial
> object given to server-side.

.. Not done, still on the drawing board. Note that If-Range must be used
here by the server side.

> ** Partial object cached. Ranged request for available ranges **
>
> Cache hit. Handled as in the case where the full object is cached.

... Not done. (we need the store backends to support sparse data first.
Currently only the memory store supports sparse data)

> ** Partial object cached. Ranged request where not all is available **
>
> Handled more or less as the "Partial object cached. Normal request" case,
> except that the requested ranges is taken into account when constructing
> the Range request.

Ditto. Drawing board etc etc.

> Ideally the request forwarding chain should be redone before doing all
> this to separate the store from the forwarding path by using a forwarding
> layer acting as a broker between client/server/store layers, with the hot
> object cache acting as a memory cache between the broker and the store.
>
>
> client side <--> forwarding layer/broker <--> server side
> ^
> |
> v
> hot object cache
> ^
> |
> v
> store
>
> this would simplify several of these issues, and allow for considerably
> more efficient operation (especially when merging partial responses).

Effectively, we have this. It's a little baroque and could be improved.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.