Re: [squid-users] Caching YouTube content

On Thu, Sep 1, 2011 at 23:05, Amos Jeffries <squid3_at_treenet.co.nz> wrote:

> On 02/09/11 06:03, Kevin Wilcox wrote:

>> On a small test deployment I can get the content to cache - and
>> retrieve - using store_url and a perl handler, at least for youtube
>> content, but the regex needs some work as it's *highly* inefficient,
>> not at all ready to scale out beyond 100 or so users (which is about
>> 5% of the population I need to handle on a single proxy) and requires
>> the URL to be *exactly* alike between viewings (you know all those
>> little&related= variations? they cause multiple downloads).
>
> What do you mean? The regex we have is supposed to erase all the volatile
> parameter pieces. So storeurl produces some fake domain with the video
> unique ID and HD resolution type as path.

Hi Amos. Sorry, the regex I referred to isn't one provided by the
project, it's the one I ended up with in my perl handler. I'll take
another look at the discussion page as I've been on another component
of the project since at least mid-summer. If my memory serves (some
three or four months later), at the time the regex from the project
was tailored for GoogleVideo and the YouTube component didn't quite
fit. I spent a few days tweaking and testing; once I had content cache
(and play properly) I noticed the perl handler would cause some
interesting CPU spikes...nothing that would cause a problem under
about 500 requests/second but it didn't inspire me to try to push it
out to a few thousand requests/second. I assume it is how I was
matching/rewriting the URL versus a performance issue with squid or
perl in general.

>> If I get it squared away I'll get some documentation and examples
>> together.
>
> YT change things every few months, we have found waiting for a full result
> is not worth it. That wiki "tutorial" is being used as a living document
> with collaborative research into the remaining problems and solutions. You
> are welcome to join in.

That's for certain, and I'm one of the worst at it. They had just
changed something that turned out to be important (thus my tweaking
and editing), I have no doubt that things are sufficiently different
now such that if I re-enabled that particular configuration it would
fail.

kmw
Received on Wed Sep 07 2011 - 18:42:25 MDT