Re: What is The logic of Vary Headers cachiness?

On 07/26/2013 02:01 PM, Henrik Nordström wrote:
> httpbis do not address mirrors.
>
> Regards
> Henrik
This is where metalinks comes to help and they do help to make object
definitions more accurate and state "this object is or a or c or b.
Or at least in the basic file level rather then having a different
MD5\SHA1 or other hash files to confirm them.

The only main issue is if there is a site that violates these headers
honor and state for a object x a hash of object y.
Which also could me a mistake that should be prevented.

So etag is fine but per server or per url.
metalink\hashes are more global.
since the list of mirrors is big and there is a list of them it would be
very simple to use StoreID helper to identify them faster then a HASH
verification.
The only problem is that there should be a computer\program friendly
list of mirrors for let say fedora..

in cases of enterprise level linux distributions mirrors should have
this kind of option.
Most of their clients are curl like and happy to "help" cache or be
"cached" to serve more people.
Once now squid do IMS then all the mirrors should have the same IMS
response for all mirrors(if they use rsync).
the public list of fedora contains currently 152 active public mirrors
with 240Gbic/sec which a squid forward proxy can pretty easily cache all
of the objects in one repo with basic settings.

Do we have friends there??
I was thinking before to use a specific mealink file they have on
specific resources to just allow the admin update the MIRROR list and
then squid will know about all the mirrors.
I will take my time later on to try compile a small mirror list which is
actually usefull and will help squid determine what and how should be
served statically.
updated DB with the examples in place.
http://wiki.squid-cache.org/Features/StoreID/DB

Notice that as we publish this feature we go from "hacking youtube" to
"allowing admins to prevent de-duplication of objects DB".
Later on we probably will be able to build such DB by metalinks headers
based DB that will allow one prefecth of http object and later on know
that this file is a specific resource that answers a or b or c or d.
This way we can cache more objects and if there is a specific request it
should be satisfied from cache if it's still there and the object is a
basic HOT or HIT but not STALE too long.(365 days object lifetime limit..)

Eliezer
Received on Fri Jul 26 2013 - 11:44:26 MDT