Re: What is The logic of Vary Headers cachiness?

On 25/07/2013 6:37 p.m., Henrik Nordström wrote:
> tor 2013-07-25 klockan 17:23 +1200 skrev Amos Jeffries:
>
>> We think alike. See my other reply. Although the lookup per-match is not
>> required. HTTP only requires that we identify a whole set of options and
>> pull the most appropriate - with some requirements around defining
>> "appropriate".
> newest match is hightly preferable.
>
>>>> Or better yet allow the same response body to be
>>>> referenced by multiple responses (simplifies validation logics and aging
>>>> considerably). Response headers is the original response updated with
>>>> header data from the 304 response in response to If-None-Match.
>>> It feels like this is kind of separate optimization. It can be
>>> restricted to Vary transactions, but does not have to be.
>> I agree. It is not clear which cases of Vary handling this would be
>> appropriate for. We do however need to fix bug #7 once and for all.
> The map used in squid-2 do not really work well in cache validations.
> The problem is that the full objects gets revalidated when we get a new
> 304 response, when actually it's only the response that matches this
> request that have been revalidated.
>
> There is also a problem with too much churn in the x-vary marker object.

Which problem specifically? that churn exists? that it can grow big +
churn? races between clients? or that letting it out to disk can cause
churn to be slooow?

I have been playing with the idea of locking these into memory cache, or
using a dedicated memory area just for them to avoid the speed issues. A
specialized store for them will also allow us to isolate the
secondary-lookup logic in that stores lookup process - it can identify
the variant and recurse down to other stores for the final selection
using the extra key bits.

I believe that they can be generated from a disk scan and if necessary
we can add swap.state TLV entries for the missing x-vary meta details to
be reloaded quickly. That would make them churn particularly badly on
startup, but avoid the necessity to store anywhere long-term, and help
detect obsolete variants undeleted from disk.

Amos
Received on Thu Jul 25 2013 - 06:54:03 MDT