Re: [squid-users] how the big guys are doing it - Caching dynamic content with squid

I don't know how others (a.k.a. the "big guys") are doing it,
but I have a suggestion:

YouTube and others sometimes change the way they redirect
a browser to different stream servers and use
different URLs for the same video, so the solution should
be configurable with a configuration file saying
how to map/match URLs to the same content.
So when a video provider changes its tricks, a user
only needs to update a config file.

Make a solution with one component: your software.
So do not make anything that relies in redirection to nginx
or others since it makes the solution more complex.

Youtube and others make videos uncacheable with
object properties saying "do-not-cache-me" and/or
using different URLs for the same video.
Simply replacing the "do-not-cache-me" (i.e.
replace the header "Cache-Control: xxx") can be
an easy solution.

The solution can be implemented with an ICAP daemon.
An ICAP daemon can do all the things that I mentioned
and is very powerful so it most likely can also provide
solutions for future tricks that YouTube and others
will use.

Marcus

On 04/25/2012 06:20 AM, x-man wrote:
> Hello,
>
> I was bugging my head recently with different approaches on how to cache
> dynamic content with squid, especially youtube content, but also other
> similar video sites.
>
> The solutions I found on the internet and tried ranged from url_rewriters
> with php scripts running on apache, to cache_peer solutions using nginx
> proxy module as cache_peer directive in squid...
>
> Some of them are working, and some of them are not working very well..
>
> I was wondering how the commercial solutions, that are based on squid also,
> are doing this, and what is the proper way to do it, if I have to start
> writing such software right now, where to start from.
>
> This questions is mostly directed to the Squid people here, who can give
> advice according to them, what is the best approach if you want to make a
> proper addon for squid, that will be dealing with dynamic content sites -
> like youtube and others similar.
>
> Of course if someone else can give idea is very much welcome in this
> discussion.
Received on Wed Apr 25 2012 - 15:16:45 MDT