Re: [squid-users] Sponsor etag/vary support for Squid 3.3

Hi Alex

Sorry, I didn't notice your reply!

>> I'm picking up an old thread from some time back. I remain interested
>> in getting support for etag into squid (and related revalidate support).
>>
>> My main requirement is that I have two proxies on either side of a
>> bandwidth limited link (with high cost). I want the situation that when
>> a client GETs some object,
> A client GETs some object currently in the cache and with ETag, but that
> cached object is either stale or being forcefully reloaded by the
> client, right?

Yes. Or some second client requests the same object so we need to do a
freshness check, or client clears their cache, or upstream doesn't
correctly implement IF-MODIFIED-SINCE, etc, etc

I'm not trying to decrease the incidence of squid asking the upstream
server if the object is fresh (which could also trigger non idempotent
changes), however, I will try to reduce the amount of bandwidth used
over the proxy-to-proxy middle link (which crosses an expensive sat
connection) by ensuring that etags are set on important resources (eg
creating one where it doesn't exist, using some hash of the content body)

What I have probably failed to consider properly is a change in headers
between two otherwise identical responses (ie same bodies), but I guess
that will become clear later.

Also I think VARY support will either drop out or be required. I have a
use in mind which would become dependent on browser version (eg serving
webp graphics to chrome)

>> we can convert this to an IF-NONE-MATCH and
>> trust the etag confirms that the object is unchanged.
>
>
>> Note, I am aware of the limitations of trusting etags. In my setup I
>> will have control over the proxy on the high speed side of the
>> connection and we can use various methods on that side to ensure that
>> the etags are sane. The main goal is to minimise bandwidth across the
>> intermediate (expensive) link.
>>
>> Previously we discussed all kinds of complex ideas including
>> implementing trailers, and custom headers with hash values. On
>> reflection I think everything required can be done using only etag
>> revalidation (and some tweaking of etags, but squid needs know nothing
>> about that...)
> Yes, reload-into-If-None-Match and stale-into-If-None-Match features
> sound simple. The latter may even be supported already (will check). If
> something outside of Squid provides reliable-enough ETags to all
> cachable responses, then the complexities discussed earlier go away.
>
> Please confirm whether my understanding of your updated requirements is
> correct.

I believe so.

So, the situation is a downstream client talking to two squid proxies in
a chain, through to the eventual upstream web server. Between the two
squid proxies is an expensive internet link (charged by the byte) and so
we desire to minimise bytes across the link.

Essentially an upstream adaption proxy will used on the "fast" (ie
"cheap") side of the connection. This will examine all responses before
they are handed to "fast side" squid and in this proxy we will beat the
etag into shape, eg adding an SHA hash if none exists, etc. Obviously I
have to accept all breakage which occurs if I change the upstream's etag
- however, I think we have this covered.

My goal is that if an object has the same response body, and it's
already in squid cache on the "slow" side of the link, then we "freshen"
the resource by going back to the origin server via our pair of squid
servers, however, we avoid the transfer of the body back across the
expensive link (between the two squid proxies) if the etag still matches

I hope this will also be useful to others than just me!

Thanks

Ed W
Received on Tue Apr 02 2013 - 17:41:56 MDT