Re: I would like to work into "caching of partial responses"

On 20/04/2012 4:40 p.m., Paolo Malfatti wrote:
> Hi, I would like to expand the Alex Rousskov's idea:
>

Have you tried contacting Shan Zhu to see where he got to. We have had
no contact since March.

>> I am not sure changing the URL is the best or even easiest way forward.
>> Instead, I would try to change how cache key is computed by adding
>> Range information to the hashing function and then adjust the "does
>> the cached store entry match the request" code to account for Range
>> request headers."
>
> Can I create a object that stores a list of range's hashes instead of
> original data?

Yes. Architecturally I would like to see the "variant pointer" object
created in store for Vary: alternatives to becomes exactly that type of
sub-index object. With a pair of (key-format, hash).

In the range cases key-format would include the URI, ETag, some keys
from the *response* values from Vary: details, and range.

NP: Fileno is tempting long-term but short-term hits the problem that
individual cache entries (fileno) can be replaced with unrelated entry
without the variant pointer being updated. hash is slow but fine for
initial use.

>
> I want to store every request individually indexed with an hash
> calculated with range, but I will keep track of them with another
> object (indexed with "normal" key) wich will maintain the list of
> ranges (and keys). That way will permit to HIT a subset of a wider
> range, or merge ranges, etc.
>
> Flow can be something like that:
>
> Client request
> range x-y
> |
> |
> V
> Store computes
> the "normal" key and
> look for it in the
> hash table
> |
> |
> V
> if the object is a HIT and it is a
> "list of ranges object" (1)
> it looks in the list for a match
> (subset, superset, whole object)
>
> |------> if any match,
> | it will retrieve the corresponding(s) object(s)
> | from disk/mem and it will send it to client
> |
> |------> if not, it will retrieve the range from the origin,
> it will store it with a "range" key and it will add it
> to the list in the (1)
>
> I really really appreciate if someone can tell me if it's a doable
> idea, or if there is a best solution.

Very doable. see also

   http://www.squid-cache.org/mail-archive/squid-dev/201203/0004.html
   http://www.squid-cache.org/mail-archive/squid-dev/201203/0001.html

Amos
Received on Fri Apr 20 2012 - 11:30:09 MDT