further to store & message flow discussion

So Henrik,
    do you have a suggestion or two as to the minimalistic changes needed to turn the "." from your ascii graphs into a separate
portion of squid, rather than being spread over several files and called inconsistently as it is today?

having realised that I _need_ this at least partially in place to finish robust support for range responses/chunked encoding and
persistent http/1.1 connections I thought we could clarify some of the ideas and then I'll get to coding. I'm delaying moving all
the rbcollins_filters code across to content_process, because I want to do the job _right_.

From your previous message:
>Yes, there is a need for a intermediary layer that collects what is
>available and constructs the appropriate upstream query and which splits
>the data flow. This layer is the . in the graphs above. But not even
>this layer theoretically needs to be very specific for HTTP. But calling
>this layer for the store is very inappropriate as it stores nothing. It
>is only logics.

As I see it the "intermediary" is visible from the server protocol side, and visible from the client protocol side, and the store is
not directly visible to either the server protocol side or the client protocol side.

If possible the intermediary layer should be protocol neutral itself, dealing in generalised messages with metadata, not http
headers and the like.

So what do we call it? Message handler? Protocol arbitrator (because it sits between the protocols of http response, http request,
store reads, store writes, ...) ?

As I see it most of the external store API calls need to be deprecated for general use and centralised into the "name to be
determined" portion of squid.

This raises another question: for internally generated objects, say gopher responses and error messages, does the intermediary
buffer them until the client reads and sends the message, or does the intermediary _appear_ to buffer them, but actually loop them
through the store as currently happens. (I have no preference here, just raising the issue).

So what we should end up with is a straightforward implementation of http server rules in client_side, http client rules in http.c,
and the logic to tie an incoming response to a request and flow the data through, copying to store if needed in the intermediary.

Design points:
* From upstream the intermediary recieves clearly marked data & metadata, and the recieved data need not be contiguous.
* For sending downstream, the intermediary handles message merging, providing client_side the requested data & metadata, or an error
condition.
* The intermediary is needed to satisfy every request because it is the interface through which client_side reaches the store
* The intermediary is not http-specific. It can deal with an incoming gopher response as accurately and effectively as an incoming
http response. Corollary: server side protocols must be able to split recieved data into metadata and message data.

Wish list (from me :] ):
* EOF indicators can be sent in-line on the last buffer, rather than afterwards. This saves a whole round of 0 byte writes through
content chains, and means that the terminating 0 on chunked transfers in client_side can be part of the final network packet, rather
than a single 5 byte network write (hardly efficient).

Rob
Received on Sun Feb 18 2001 - 05:51:43 MST